Copying an idea from mimalloc, initialise the TLS variable to a global
allocator that doesn't own any memory and then lazily check when we hit
a slow path (which we always do when using the global allocator, because
it doesn't own any memory) if we are the global allocator and replace
it.
There is a slight complication compared to mimalloc's version of this
idea. Snmalloc collects outgoing messages and it's possible for the
first operation in a thread to be a free of memory allocated by a
different thread. We address this by initialising the queues with a
size value indicating that they are full and then do the lazy check when
about to insert a message that would make a queue full. This will then
trigger lazy creation of an allocator.
Global initialisation doesn't work for the fake allocator, so skip most
of its constructor.
This change introduces a per small sizeclass free list. That can be
used to access the free objects for that sizeclass with minimal
calculations being required.
It changes to a partial bump ptr. We bump allocate a whole OS
page worth of objects at a go, so we don't switch as frequently
between bump and free list allocation.
The code for the fast paths has been restructured to minimise the
work required on the common case, and also it is all inlined for the
common case.
Allocating a zero sized object is moved off the fast path. Ask for 1
byte if you want to be fast.
This is useful as codegen is nicer if we use size_t, but the semantics
is uint8_t, and is stored as that in many places in the metadata.
Ultimately should introduce a wrapper to check this invariant.
Will detect corruption caused by either
* Use-after-free
* Double-free
Neither is comprehensive. Full temporal safety is not possible.
This just aids with debugging.
This introduces a new `address_t` type and two new casts: `pointer_cast`
and `address_cast` for casting between an `address_t` and a pointer.
These should make it easier to audit the codebase for casts between
pointers and integers. In particular, the remaining `reinterpret_cast`s
and `pointer_cast`s should be the only places where we could perform
invalid pointer arithmetic.
Also adds a `pointer_offset` helper that adds an offset (in bytes) to a
pointer, preserving its original type. This is a sufficiently common
pattern that it seemed worthwhile to centralise it.
We don't follow the pointers passed into the pagemap directly, but
instead use them to calculate indexs into the pagemap. Use uintptr_t
means it is easier to perform address arithmetic, and not have casts
back to void* everywhere.
Rather than unconditionally storing a pointer to the pagemap and
initialising this with a global, we now provide a choice of three
compile-time options for how the pagemap should be accessed.
If you're creating a new allocator and the pagemap comes from a library
that exports the pagemap accessor function but not the pagemap symbol,
you need to be able to replace this.
This does not deallocate memory until the OS tells us that we are short
on memory, then tries to decommit all of the cached chunks (except for
the first page, used for the linked lists).
Nowhere near enough testing to commit to master yet!
