There are two things calling themselves pagemaps:
- the src/mem/pagemap.h objects of that name
- the SuperslabMap object gets called a PageMap inside the Allocator
Rename the latter to chunkmap, with appropriate case and snake,
everywhere, and pull it out to its own file (chunkmap.h).
The default implementation of a chunkmap is a purely static object, but
we nevertheless instantiate it per allocator, so that other
implementations can use stateful instances when interposing on the
mutation methods. Note that the "get" method, however, must remain
static to support the interface required by Allocator objects.
We know that these are small objects and that they have already had
their cache friendly offsets applied, so jump in to the dealloc path
futher down, below the stats calls.
Split small_dealloc_offseted into a wrapper around the core to avoid
inlining too much into the debug function.
While here, move the handling of the Remote objects into its own block
so that `p` is out of scope thereafter.
Fixes https://github.com/microsoft/snmalloc/issues/98
Debug_check_empty now empties the free lists, and checks it empties all the
queues in the allocator. This does not require statistic tracking to
work anymore.
This additionally can check internal regression that cause leaks that
are not the clients fault.
If the first call to alloc uses the minimum size class, then we end up
leaking a whole page of allocations. We fill the small_fast_free_list,
in a call to build the message_queue. But this means the
small_fast_free_list[0] is not empty. But the code was staying on the
slow path, and overwriting it.
If we are using USE_MALLOC pass through, then ThreadAlloc::get() can
cause an allocation inside snmalloc still, as it builds the thread
local allocator, which allocs a single stub allocator, using
the underlying allocator. This means that ASAN would detect a
leak in a client, even though the client program has none.
This change stops us ever allocating allocators, if we are passing
through calls to an underlying allocator.
HEADER_GLOBAL was using non-standard attributes to achieve what C++17
now permits with a keyword. Use the standard formulation.
Update the README to note that gcc is still not recommended, but because
of its poor codegen for 128-bit atomic compare and exchange, rather than
because it doesn't support the attribute used for HEADER_GLOBAL.
The pagemap global is now an inline static of a template class, so that
we will see different symbols for the different types.
Issue #84 showed that it's possible to compile two compilation units
with different pagemaps, link them together, and have them attempt to
interpret the global pagemap as two different types. This change should
make that impossible.
Also make the `pagemap()` function static so that it can be used from
static functions, avoiding other things that directly reference the
global pagemap.
Made the API so that get always returns an initialised Alloc*. Added
new fast path that doesn't perform checking, but can lead to very slow
behaviour if called and reused.
If the operating system will allocate private pages on demand for the
pagemap then use the FlatPageMap by default as it generates better code
for deallocation.
This commit changes the strategy for finding a free list from
a stack to a queue. This tends to avoid the slow path considerably more.
It has some memory overheads.
TOOD: We should move the bump allocation data out of the metaslab and
into the allocator. At the moment, the slab contains the bump allocation
data, we should move this into the allocator, as it only ever has one slab
it is bump allocating from per sizeclass.
This is needed because in some configurations the constructor for the
global placeholder is not called before the first allocation (i.e. when
other globals call the allocator in their constructor) and so we ended
up following a null pointer.
Most compilers are happy if you say always-inline but they can't. GCC
will complain. Here, we have two mutually recursive functions that are
marked as always inline. In an optimised build, one is inlined into the
other and then becomes a tail-recursive function that should inline the
tail call. Inlining the tail call can be done by simply jumping to the
start of the function and so everything is fine. In a debug build, the
second transform doesn't happen and so we're left with a call to an
always-inline function.
