* NFC: split freelist_queue from remoteallocator
This lets us use freelists as message queues in contexts other than
the remoteallocator. No functional change indended.
* freelist_queue: add and use destroy_and_iterate
* freelist: make backptr obfuscation key "tweakable"
* freelist: tweakable keys in forward direction, too
* test/perf/msgpass: ubench a producer-consumer app
Approximate a message-passing application as a set of producers, a set of
consumers, and a set of proxies that do both. We'll use this for some initial
insight for https://github.com/microsoft/snmalloc/issues/634 but it seems worth
having in general.
This provide a way to configure snmalloc to provide per object meta-data that is out of band. This can be used to provide different mitigations on top of snmalloc, such as storing memory tags in a compressed form, or provide a miracle pointer like feature.
This also includes a couple of TSAN fixes as it wasn't fully on in CI.
* Move sizeclass debugging code to sizeclass test
The sizeclass was already testing most of this, so just add the missing bits.
Forgo some tests whose failure would have implied earlier failures.
This moves the last dynamic call of size_to_sizeclass_const into tests
(and so, too, to_exp_mant_const). sizeclasstable.h still contains a static
call to compute NUM_SMALL_SIZECLASSES from MAX_SMALL_SIZECLASS_SIZE.
* Remove unused to_exp_mant
Only its _const sibling is used, and little at that, now that almost everything
to do with sizes and size classes is table-driven.
* test/memcpy: trap, if we can, before exiting
This just means I don't need to remember to set a breakpoint on exit
* test/memcpy: don't assume sizeclass 0 is allocable
* test/memory: don't assume sizeclass 0 is allocable
* test/sizeclass: handle nonzero minimum sizeclasses
* sizeclass: distinguish min alloc and step size
Add support for a minimum allocation size that isn't the minimum step of
the sizeclass table.
* Expose MIN_ALLOC_{,STEP}_SIZE through cmake
* test/sizeclass: report MIN_ALLOC_{STEP_,}SIZE
If we're running with the freelist_backward_edge mitigation turned on, then
we're going to follow the pointer, not just de-obfuscate it (in freelist's
atomic_read_next), so even if the queue heads are tame, we still need to
do this domestication.
* Template construction of Pool elements
The Pool class is used by verona-rt. The recent changes made this
less nice to consume as an API.
This change makes the construction logic a template parameter to the
Pool. This enables standard allocation to be used from Verona.
* Drop parameter from acquire
Pool::acquire took a list of parameters to initialise the object that it
constructed. But if this was serviced from the pool, the parameter
would be ignored. This is not an ideal API.
This PR removes the ability to pass a parameter.
* Benchmark for testing startup performance.
* Make pool pass spare space to pooled item
The pool will result in power of 2 allocations as it doesn't have a
local state when it is initially set up.
This commit passes this extra space to the constructor of the pooled
type, so that it can be feed into the freshly created allocator.
Co-authored-by: Nathaniel Wesley Filardo <nfilardo@microsoft.com>
This allows them to exist as fields without invalidating the set.
This should make it possible to remove the undefined behaviour in the
creation of FrontendSlabMetadata, which is currently created via a
reinterpret_cast from a different-typed allocation.
FrontendSlabMetadata has a SeqSet::Node field that is in an unspecified
state on construction and which is valid only when inserted into a
SeqSet.
* Fix#631
Add wrapper override for the Windows variant of maloc_usable_size : _msize
Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
* Using exclusive mode prefetch
The prefetching is always used to move the cache line to the current
core for writing. This change makes it use exclusive mode prefetch
and enables it as a feature flag for x64.
* Debug platform for BSDs
* CI fixes
* More CI
* Update ARM prefetch
* Update x64 prefetch default
* Factor out libc code into a header.
This pulls the main definitions of the various libc malloc functions
into a header for easier use and inclusion in other projects.
* Clang-tidy fixes.
* Clang-tidy fixes really.
* More code quality changes
* Minor fix
* Clangformat
The current version requires clang-format-9. This now getting hard to get.
This commit moves it to the clang-format-15, which is the latest in 22.04.
Also, updates clang-tidy to 15 as well.
* Make a conditional range
This range allows for a contained range to be disabled at runtime.
This allows for thread local caching to be disabled if the initial fixed
size heap is below a threshold.
There was a mis-compilation in a Verona configuration that lead to
two instances of key_global existing. This change moves it inside
a struct that seems to fix the issue.
The rest of the changes are limiting the use of key_global as both
RemoteCache and RemoteAllocator must use the same configuration,
so there is no need to take the key_global as a parameter.
The buddy allocator doesn't need to look at sizes above the current
highest size. This commit tracks the highest block that is stored in the
buddy allocator.
All the checks and mitigations have been placed under feature flags.
These can be controlled by defining
SNMALLOC_CHECK_CLIENT_MITIGATIONS
This can take a term that represents the mitigations that should be enabled.
E.g.
-DSNMALLOC_CHECK_CLIENT_MITIGATIONS=nochecks+random_pagemap
The CMake uses this to build numerous versions of the LD_PRELOAD library and
tests to allow individual features to be benchmarked.
Co-authored-by: Nathaniel Wesley Filardo <nfilardo@microsoft.com>
When using dlopen with RTL_DEEPBIND the LD_PRELOAD used
by the majority of allocators does not work as both libc
and snmallocs allocators can be called by various dso.
This patch uses GLIBC's __malloc_hook to override the allocator
as well as LD_PRELOAD. This means that all libraries will
call snmalloc when performing allocation.
* Move Morello CI to track default release
- Log some details of the build environment
- Remove workarounds overcome by events
* Morello CI: parameterize run queue and boot env
* Morello CI to run as a non-root user
For reasons unrelated to snmalloc, it's become more convenient to engage
in a little white lie, as it were, that the CI jobs are not `root` on
the worker nodes. So I'm testing changes on the cluster orchestration
goo to run the github runner as a non-root user. However, much as with
GitHub's own runners, the runner user is in the `wheel`, and `root` will
have no password, so we can still `su` up to `root` when needed.
Of course, when we are already root, we can `su` to anyone we like,
including `root`, so these changes are compatible with both the old and
new world order and have been tested with both.
This uses VirtualBox instead of xhyve. It might be slower, but should
be more reliable.
Tests run on FreeBSD, NetBSD, and OpenBSD. Only the FreeBSD ones are
passing at the moment, the others will keep running but aren't added as
dependencies for the action used to guard commits.
Compiler versions do not imply standard library versions, and even where
the compiler and standard library versions were matched, this check was
wrong.
