Commit Graph

26 Commits

Author SHA1 Message Date
Paul Liétar
a5379b24d5 Add remove_cache_friendly_offset calls in a few places that were missing it. 2019-07-17 13:14:59 +01:00
David Chisnall
2efcddfc3d Rework free list so that 0 is the placeholder.
This is needed because in some configurations the constructor for the
global placeholder is not called before the first allocation (i.e. when
other globals call the allocator in their constructor) and so we ended
up following a null pointer.
2019-07-05 14:20:24 +01:00
Matthew Parkinson
eb4e28e8d0 CR Feedback
Removed stub from message queue, and use an actual allocation.
2019-07-02 14:08:05 +01:00
Matthew Parkinson
621b7e6b9a Clang format. 2019-07-02 10:51:18 +01:00
Matthew Parkinson
7a198cbda5 Aggressively optimise fast path for allocation
This change introduces a per small sizeclass free list.  That can be
used to access the free objects for that sizeclass with minimal
calculations being required.

It changes to a partial bump ptr.  We bump allocate a whole OS
page worth of objects at a go, so we don't switch as frequently
between bump and free list allocation.

The code for the fast paths has been restructured to minimise the
work required on the common case, and also it is all inlined for the
common case.

Allocating a zero sized object is moved off the fast path.  Ask for 1
byte if you want to be fast.
2019-07-01 14:35:35 +01:00
Matthew Parkinson
cc6a9775d6 Made checks on client have own macro. 2019-06-12 15:29:25 +01:00
Matthew Parkinson
be1898c4e8 Fixed warning. 2019-06-12 14:12:47 +01:00
Matthew Parkinson
88d9534453 CR Feedback + clangformat 2019-06-12 13:54:14 +01:00
Matthew Parkinson
a8dd065fd7 A few checks for easy corruptions
Will detect corruption caused by either
* Use-after-free
* Double-free
Neither is comprehensive.  Full temporal safety is not possible.
This just aids with debugging.
2019-06-06 13:31:25 +01:00
Matthew Parkinson
ef18d7d8d5 Made initialization of slab clearer. 2019-05-21 15:25:33 +01:00
Matthew Parkinson
7de46182a8 Used start of a slab for link. 2019-05-21 09:47:23 +01:00
David Chisnall
8f082c6a9c Fix incorrect edit in earlier commit. 2019-04-30 14:11:15 +01:00
David Chisnall
8bd136c224 Remove all instances of static_cast<size_t>(1).
In preparation for removing magic numbers, simplify code that is
computing masks with a single bit set.
2019-04-30 11:50:32 +01:00
David Chisnall
5c197e4ae4 [NFC] More checks, comments on end of namespace braces. 2019-04-30 09:46:01 +01:00
David Chisnall
28fac4d700 Fix the remaining clang-tidy warnings.
This introduces a new `address_t` type and two new casts: `pointer_cast`
and `address_cast` for casting between an `address_t` and a pointer.
These should make it easier to audit the codebase for casts between
pointers and integers.  In particular, the remaining `reinterpret_cast`s
and `pointer_cast`s should be the only places where we could perform
invalid pointer arithmetic.

Also adds a `pointer_offset` helper that adds an offset (in bytes) to a
pointer, preserving its original type.  This is a sufficiently common
pattern that it seemed worthwhile to centralise it.
2019-04-29 13:37:05 +01:00
David Chisnall
4bafca9be7 [NFC] Automatic fixes from clang-tidy. 2019-04-29 11:33:07 +01:00
David Chisnall
365571b2b8 Add an explicit invalid pointer type.
This is a generalisation of `std::nullptr_t` that allows non-0 values of
pointer-sized-but-not-a-pointer thing.  `DLList` now takes an
`InvalidPointer`, `nullptr_t`, or some other compatible class as a
sentinel value instead of a `uintptr_t`.
2019-04-29 11:22:40 +01:00
Matthew Parkinson
f1835813bd uintptr_max instead of -1/~0 2019-04-29 11:02:47 +01:00
Paul Liétar
ffe496dd18 Address review comments. 2019-02-18 21:29:32 +00:00
Paul Liétar
4ac3421487 Place the next pointer at a different place on every object.
This limits the collision in cache buckets, especially for larger objects.
2019-02-18 14:22:50 +00:00
Matthew Parkinson
ec59820970 Use placement new instead of init for metaslab. 2019-01-25 19:46:24 +00:00
Matthew Parkinson
06ff1ffe0a Renamed zero to init for metaslab. 2019-01-25 16:38:10 +00:00
Matthew Parkinson
294efe8e03 Hardening access to meta data
Introduce a wrapper that ensure various fields of meta data, represent
valid indexes into the the address space, and not beyond the current
slab/superslab.
2019-01-24 11:39:18 +00:00
Matthew Parkinson
e30c94cb2a Apply ModArray to Metaslab data
Prevent bugs accessing outside the Slabs meta data.
2019-01-24 11:39:18 +00:00
Matthew Parkinson
94f8b886a0 Remove union in metaslab
The union in Metaslab provides no benefit in size, as the single byte
it effectively saves will be removed due to padding.

By removing the union, we get stronger properties over sizeclass, and
remove an out-of-bounds access.
2019-01-18 21:11:01 +00:00
Matthew Parkinson
4f9d991449 Initial commit of snmalloc
History squashed from internal development.

Internal history has commit hash:
  e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 14:17:55 +00:00