* Use C++17 inline statics
This leads to better codegen in GCC, and fixes some linking issues in OE.
* Detect GCC and OE combination and fall-back to lock based ABA.
* clangformat
* Remote dealloc refactor.
* Improve remote dealloc
Change remote to count down to 0, so fast path does not need a constant.
Use signed value so that branch does not depend on addition.
* Inline remote_dealloc
The fast path of remote_dealloc is sufficiently compact that it can be
inlined.
* Improve fast path in Slab::alloc
Turn the internal structure into tail calls, to improve fast path.
Should be no algorithmic changes.
* Refactor initialisation to help fast path.
Break lazy initialisation into two functions, so it is easier to codegen
fast paths.
* Minor tidy to statically sized dealloc.
* Refactor semi-slow path for alloc
Make the backup path a bit faster. Only algorithmic change is to delay
checking for first allocation. Otherwise, should be unchanged.
* Test initial operation of a thread
The first operation a new thread takes is special. It results in
allocating an allocator, and swinging it into the TLS. This makes
this a very special path, that is rarely tested. This test generates
a lot of threads to cover the first alloc and dealloc operations.
* Correctly handle reusing get_noncachable
* Fix large alloc stats
Large alloc stats aren't necessarily balanced on a thread, this changes
to tracking individual pushs and pops, rather than the net effect
(with an unsigned value).
* Fix TLS init on large alloc path
* Add Bump ptrs to allocator
Each allocator has a bump ptr for each size class. This is no longer
slab local.
Slabs that haven't been fully allocated no longer need to be in the DLL
for this sizeclass.
* Change to a cycle non-empty list
This change reduces the branching in the case of finding a new free
list. Using a non-empty cyclic list enables branch free add, and a
single branch in remove to detect the empty case.
* Update differences
* Rename first allocation
Use needs initialisation as makes more sense for other scenarios.
* Use a ptrdiff to help with zero init.
* Make GlobalPlaceholder zero init
The GlobalPlaceholder allocator is now a zero init block of memory.
This removes various issues for when things are initialised. It is made read-only
to we detect write to it on some platforms.
Change the result of `operator=` on `Mod` to return `Mod&`. In this
example:
```c++
Mod<4, int> a;
int b = (a = 8);
```
The value of `b` will now be 0, not 8, and will equal the value of `a`.
Hopefully this is less confusing to users of this class.
This introduces a new `address_t` type and two new casts: `pointer_cast`
and `address_cast` for casting between an `address_t` and a pointer.
These should make it easier to audit the codebase for casts between
pointers and integers. In particular, the remaining `reinterpret_cast`s
and `pointer_cast`s should be the only places where we could perform
invalid pointer arithmetic.
Also adds a `pointer_offset` helper that adds an offset (in bytes) to a
pointer, preserving its original type. This is a sufficiently common
pattern that it seemed worthwhile to centralise it.
Introduce a wrapper that ensure various fields of meta data, represent
valid indexes into the the address space, and not beyond the current
slab/superslab.
If a sizeclass in the metadata is corrupted, then this can be used to
force an index beyond the end of these tables. This extends the tables
to the next power of two, and uses a mask on the index, so they are
always either a valid piece of data, or zero.
