MetaCommon is now gone. The back end must provide a SlabMetadata,
which must be a subtype of MetaSlab (i.e. MetaSlab or a subclass of
MetaSlab). It may add additional state here.
The MetaEntry is now templated on the concrete subclass of MetaSlab that
the back-end uses. The MetaEntry still stores this as a `uintptr_t` to
allow easier toggling of the boundary bit but the interfaces are all in
terms of stable types now.
Also some tidying of names (SharedStateHandle is now called Backend).
In a follow-on PR, we can then remove the chunk field from the
BackendMetadata in the non-CHERI back end and allow back ends that don't
require extra state to use MetaSlab directly.
Other cleanups:
- Remove backend/metatypes, define the types that the front end expects
in mem/metaslab. The back end may extend them but these types define
part of the contract between the front and back ends.
- Remove FrontendMetaEntry and fold its methods into MetaEntry.
- For example purposes, the default back end now extends MetaEntry.
This also ensures that nothing in the front end depends on the
specific type of MetaEntry.
- Some things now have more sensible names.
The meta entry now operates in one of three modes:
- When owned by the front end, it stores a pointer to a remote, a
pointer to some MetaSlab subclass, and a sizeclass.
- When owned by the back end, it stores two back-end defined values
that must fit in the bits of `uintptr_t` that are not reserved for
the MetaEntry itself.
- When not owned by either, it can be queried as if owned by the front
end.
The red-black tree has been refactored to allow the holder to be a
wrapper type, removing all of the Holder* and Holder& uses and treating
it uniformly as a value type that can be used to access the contents.
The chunk field is fone from the slab medatada.
This will need to be added back in the CHERI back ends, but it's a
back-end policy. The back end can choose to use it or not, depending on
whether it can safely convert between an Alloc-bounded pointer and a
Chunk-bounded pointer.
The term 'metaslab' originated in snmalloc 1 to mean a slab of slabs.
In the snmalloc2 branch it was repurposed to mean metadata about a
slab. To make this clearer, all uses of metaslab are now gone and have
been renamed to slab metadata. The frontend metadata classes are all
prefixed Frontend and some extra invariants are checked with
`static_assert`.
David points out that the downcasts I had introduced were UB. Instead, go back
to passing MetaEntry-s around and make MetaslabMetaEntry just a namespace of
static methods.
This partially reverts 7940fee00c
Otherwise these won't get updated until the small buddy allocator hands them off
to the large buddy allocator (when they morph into being rbtree nodes) and so
the frontend might get confused in the interim (including risk of UAF on
double-free).
These are almost entirely backend concerns, so move their definitions over
there. Use C++ friend classes to ensure that MetaCommon structures are opaque
to frontend code (at least, at compile time, and neglecting the rest of C++).
(These structures contain high-authority pointers and so should be as closely
guarded as we can make them.)
The bits that leak out are
- the encoding of RemoteAllocator* and sizeclass_t into the uinptr_t within a
MetaEntry. This, however, is almost entirely a frontend concern, so detach
the method definitions from the class and leave those in mem/metaslab.h for
the moment.
- the size of metadata structures pointed to by the MetaEntry meta field.
Rather than use sizeof(Metaslab) (and assert that sizeof(ChunkRecord) is
smaller), instead, define PAGEMAP_METADATA_STRUCT_SIZE once and assert that
all records fit. Additionally, add an assertion that Metaslab is exactly this
size, not for semantic reasons, but because we expect it to be true.
The bits that leak in are
- the need to zero memory corresponding to a chunk. Rather than having an
escape hatch that reveals the MetaCommon.chunk, move the zeroing call into a
small wrapper method within the MetaCommon class itself.
- the need to get the address of a chunk. We want to assert that we've got the
right chunk on occasion (well, at least once so far) and so add a class method
to expose the address_t view of the chunk pointer without exposing the pointer
itself.
# Small changes before rewrite
* Additional bit in remote allocator to prevent type confusion with the backend.
* Move Chunk allocator to backend.
* Improvements to RedBlack tree
* Expose message from Pal
# Complete backend rewrite
This provides two key changes:
* We use buddy allocators to allow memory to reconsolidated
* The backend is factored into a series of small operations that
allocate and deallocate memory.
The backend now uses "Ranges", there are two ranges that don't require a
parent range:
* EmptyRange - Never returns any memory
* PalRange - Returns memory from the platform.
All other ranges require a parent range to supply memory to them. Some
ranges support both allocation and deallocation, and some just
deallocation. For instance, CommitRange supports both, and maps
requests to the parent range, but will Commit and Decommit the memory.
As the ranges perform only a single task, they are generally small and
easy to follow. The two exceptions to this are the two BuddyRanges
(Large and Small). Large is for CHUNK_SIZE and above blocks, while
Small is for below CHUNK_SIZE blocks. Both are implemented with a buddy
allocator, but the SmallBuddyRange uses in place meta-data, while the
LargeBuddyRange uses the pagemap for its meta-data. This means the
LargeBuddyRange can keep the majority of memory it is managing
decommitted.
The Backend glues together the various ranges to support the appropriate
way to manage memory on the platform.
* Add default for getting chunk allocator state
Makes the API same between the two configurations.
* Reduce address space usage for Open Enclave
* Fix OE Pal concept
* Add support for Pal not to provide time.
The lazy return of pages to the OS uses a simple time
based heuristic. This enables a PAL to not support time,
and return the memory to a central pool immediately.
* Update src/backend/backend.h
Co-authored-by: Amaury Chamayou <amaury@xargs.fr>
Co-authored-by: Amaury Chamayou <amaury@xargs.fr>
This preserves the chunk pointer through the use of a chunk as a slab. It does
grow the structure by one pointer, but on non-CHERI it is still padded to 64
bytes, even with CHECK_CLIENT guards in place:
0: MetaCommon chunk pointer
8: next pointer
16: builder head[0]
24: builder head[1]
32: builder tail[0]
40: builder tail[1]
48: builder length[0] (uint16_t)
50: builder length[1] (uint16_t)
52: padding (4 bytes)
56: needed (uint16_t)
58: sleeping (bool)
(Sadly, on CHERI, even without CHECK_CLIENT guards and with no padding, there
are now four pointers in the structure -- chunk, next, head, tail -- plus five
extra bytes. We will likely wish to explore encoding the head and tail offsets
relative to the chunk pointer.)
This lets us remove the "subversive amplification" in dealloc() in favor of just
preserving the chunk pointer. Speaking of, be sure to assign that in all the
right places, and ASSERT that we've got it right.
`capptr_domesticate<Backend>(Backend::LocalState*, CapPtr<T, B>)` is the
intended affordance for conversion to covert from a `CapPtr<T, B>` with
`B::wildness` `Wild` to a `CapPtr<>` with `B::with_wildness<Tame>` and thence
plumbed into the rest of the machinery.
David added the SFINAE wrapper so that `Backend`-s now don't need to implement a
domestication callback; instead, if the `Backend` does not provide a
`capptr_domesticate` function, a default, which just does an explicit type cast,
will be used instead.
This is not yet hooked into the rest of the tree.
Co-authored-by: David Chisnall <David.Chisnall@microsoft.com>
* Switch to a multidimensional taxonomy.
Rather than encoding the abstract bound states in a single enum, move to a
more algebraic treatment. The dimensions themselves are within the
snmalloc::capptr_bounds namespace so that their fairly generic names do not
conflict with consumer code. Aliases for many points in the space are
established outside that namespace for ease of use elsewhere.
* Introduce several new namespaces:
* snmalloc::capptr::dimension holds each of the dimension enums
* snmalloc::capptr holds the bound<> type itself and a ConceptBound
* snmalloc::capptr::bounds gives convenient specializations of bound<>
* snmalloc::capptr also has aliases for CapPtr<> itself
All told, rather than `CapPtr<T, CBChunk>`, we now expect client code to read
`capptr::Chunk<T>` in almost all cases (and this is just an alias for the
appropriate `CapPtr<T, bounds<...>>` type). When the bound<>s themselves are
necessary, as when calling capptr_bound, we expect that they will almost
always be pronounced using an alias (e.g., `capptr::bounds::Alloc`).
* Chase consequences.
* Prune old taxa and aliases that are no longer in use in snmalloc2.
David points out that we might not have a static way to get at the pagemap, so
it is potentially useful to pass pointers to state objects down from the
Allocators.
And do so by type, rather than by value. While here, introduce a C++20 concept
for this Backend-offered proxy and adjust the template parameters appropriately.
This will be useful for the process sandbox code, which needs to mediate stores
to the pagemap, but can provide a read-only view.
This is the set of changes required for snmalloc2 to be usable by the
process sandboxing code and incorporates some API changes that reduce
the amount of code required to embed snmalloc. Highlights:
- Merge the config and back-end classes.
- Everything in config is now global (all methods are static)
- The GlobalState class is gone (all global state is managed by global
methods on the config class)
- LocalState is now a member of the config class, all methods are
instance methods.
- Not every configuration needs to use the lazy initialisation hooks.
They now need to be provided only if they are used. If the
configuration does not provide an `ensure_init` method, it is not
called. If it does not provide an `is_initialised` method then the
global initialisation state is not checked.
- There is now an `snmalloc::Options` class that default initialises
itself to the default behaviour. Every configuration must provide a
`constexpr` instance of this class. Each flag can be separately
overridden and new flags can be added without breaking any existing
API consumers.
The config classes are moved into the backend directory.
* Make address space manager use pagemap for next pointers
The address space manager uses the pagemap entry to form linked lists of
unused address space above MIN_CHUNK_SIZE. It continues to use
references in the block below that threshold.
In the CHECK_CLIENT mode this makes it hard to corrupt the ASM as only
meta-data uses allocations below MIN_CHUNK_SIZE from the ASM. This
allocations will be protected with guard pages by the backend.
* address_space_core: use FreeChunk struct
Purely stylistic, NFCI. This hides some somewhat gnarly reinterpret_cast-s in
favor of more, but hopefully less gnarly, casts elsewhere.
* Apply suggestions from code review
Co-authored-by: Nathaniel Wesley Filardo <nfilardo@microsoft.com>
The meta-data in the CHECK_CLIENT mode is allocated from its own areas,
where most of the pages have been disabled, and the location within this
range is randomised.
This protects from trivial OOB read/write hitting the meta-data.
More complex controlled offsets are also mitigated due to the sparse
level of pages being turned on (i.e. not PROT_NONE).
The Pal was providing policy for overallocating a block of memory to
achieve alignment make that part of the backend.
The backend should be responsible for layout policy.
This PR exposes a pagemap interface to specify ranges that are being
used. The overall invariant is that any memory in the address space
manager has the pagemap committed. This means that individual operations
do not need to commit entries.
This is important for Windows that does not support lazy commit. It is
also important if we want to PROT_NONE most of the pagemap to reduce the
risk of memory safety issues getting access to the pagemap.
There are minor changes to test to pull memory directly from the Pal.
There are also bug fixes in the pagemap tests.
The pagemap allocates it self directly either from
* the original fixed address range it is supplied, and returns the
remaining space after the pagemap is removed; or
* directly allocated from the PAL without using the address space
manager.
This change in layering is required for the next commit, which imposes
the invariant that the pagemap has been committed for all spaced managed
by the address space manager.
# Pagemap
The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations.
This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation.
# Backend
There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways.
# Thread local state
The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled.
# Difference
This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
