See src/snmalloc/README.md for an explanation of the layers.
Some other cleanups on the way:
Fine-grained stats support is now gone.
It's been broken for two years, it depends on iostream (which then
causes linker failures with libstdc++) and it's collecting the wrong
stats for the new design. After discussion with @mjp41, it's better to
remove it and introduce new stats support later, rather than keep broken
code in the main branch.
Tracing was controlled with a preprocessor macro, now there's also a
CMake option.
* Post large deallocations to original thread
This change sets all large allocations to be owned by the originating
thread. This means they will be messaged back to the original thread
before they can be reused.
The following reason for making this change:
* This will improve producer/consumer apps involving large allocations.
* It enables the implementation of a more complex chunk allocator that
reassembles chunks.
* It addresses an issue with compartmentalisation where the handling of
large allocations can result in meta-data ownership changing.
The primary aim for this refactor is to use a representation for
sizeclasses that uniformly covers both large and small. This allows
certain operations such as alloc_size and external_pointer to be
uniformly implemented.
The additional types make clear which kind of sizeclass is in use.
This also tidies up the code for sizeclass based divisible by and
modulus.
It fixes a bug in rust_realloc that didn't correctly determine a realloc
was required for large classes.
# Pagemap
The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations.
This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation.
# Backend
There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways.
# Thread local state
The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled.
# Difference
This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
The previous reciprocal division branch on the prime that the sizeclass
was constructed from. All sizeclasses can be represented as
2^n * {1,3,5,7}
This lead to a very small table, but some work to calculate the
appropriate shifts and multiplications to implement reciprocal division.
This commit uses a completely uniform representation for every
sizeclass using a lookup table. Due to the precise ranges that we query
the modulus and rounding on, we can do this much more efficiently.
The func-release-rounding exhaustively tests all the queries we are
interested in.
- Don't run an expensive functionality test in debug builds.
- Don't run the different cache configurations (they're probably going
away soon because they help only in synthetic benchmarks).
