These statistics can be maintained with effectively zero cost to
realistic applications. They do not track the precise amount of
memory used, but are an over-approximation.
This change makes the original 16MiB option not the common option.
It also changes the names of the defines to
SNMALLOC_USE_LARGE_CHUNKS
SNMALLOC_USE_SMALL_CHUNKS
The second should be set for Open Enclave configuration, and results in
256KiB chunk sizes. The first being set builds the original 16MiB chunk
sizes. If neither is set, then we default to 1MiB chunk sizes.
PALOpenEnclave object is lazily constructed. I couldn't
figure out a straight-forward way to pass the heap bounds to
the constructor of PALOpenEnclave object.
As an alternative, store the bounds in inline static variables of
the PALOpenEnclave class and set them via static setup_initial_range
function.
- two_alloc_types/alloc1.cc
Define oe_allocator_init to forward base, end values to
PALOpenEnclave::setup_inital_range
- two_alloc_types/main.cc
Use oe_allocator_init function to set up heap range.
- fixed_region/fixed_region.cc
Initialize heap range via call to PALOpenEnclave::setup_inital_range.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* Improved malloc style tests
Added comprehensive testing of realloc, and other minor improvements
to reporting errors.
* Fix realloc resizing for large sizeclasses.
The rounding by sizeclass was incorrect for large allocation. This fixes
that.
* Ensure alloc_size is committed
There is an awkward interaction between alloc_size and
committing only what is requested. If the user assumes
everything up to alloc_size is available, then we need to
either store the more precise size for alloc_size to return
or commit the whole 2^n range, so that alloc_size stays simple.
This changes to just make the whole range committed.
In the future, we might want to store a more precise size, so
that the allocation can be sized more precisely.
* Reduce size of objects.
* Defensive code for alloc/dealloc during TLS teardown
If an allocation or deallocation occurs during TLS teardown, then it is
possible for a new allocator to be created and then this is leaked. On
the mimalloc-bench mstressN benchmark this was observed leading to a
large memory leak.
This fix, detects if we are in the TLS teardown phase, and if so,
the calls to alloc or dealloc must return the allocator once they have
perform the specific operation.
Uses a separate variable to represent if a thread_local's destructor has
run already. This is used to detect thread teardown to put the
allocator into a special slow path to avoid leaks.
* Added some printing first operation to track progress
* Improve error messages on posix
Flush errors, print assert details, and present stack traces.
* Detect incorrect use of pool.
* Clang format.
* Replace broken LL/SC implementation
LL/SC implementation was broken, this replaces it with
a locking implementation. Changes the API to support LL/SC
for future implementation on ARM.
* Improve TLS teardown.
* Make std::function fully inlined.
* Factor out PALLinux stack trace.
* Add checks for leaking allocators.
* Add release build of Windows Clang
* Remote dealloc refactor.
* Improve remote dealloc
Change remote to count down to 0, so fast path does not need a constant.
Use signed value so that branch does not depend on addition.
* Inline remote_dealloc
The fast path of remote_dealloc is sufficiently compact that it can be
inlined.
* Improve fast path in Slab::alloc
Turn the internal structure into tail calls, to improve fast path.
Should be no algorithmic changes.
* Refactor initialisation to help fast path.
Break lazy initialisation into two functions, so it is easier to codegen
fast paths.
* Minor tidy to statically sized dealloc.
* Refactor semi-slow path for alloc
Make the backup path a bit faster. Only algorithmic change is to delay
checking for first allocation. Otherwise, should be unchanged.
* Test initial operation of a thread
The first operation a new thread takes is special. It results in
allocating an allocator, and swinging it into the TLS. This makes
this a very special path, that is rarely tested. This test generates
a lot of threads to cover the first alloc and dealloc operations.
* Correctly handle reusing get_noncachable
* Fix large alloc stats
Large alloc stats aren't necessarily balanced on a thread, this changes
to tracking individual pushs and pops, rather than the net effect
(with an unsigned value).
* Fix TLS init on large alloc path
* Add Bump ptrs to allocator
Each allocator has a bump ptr for each size class. This is no longer
slab local.
Slabs that haven't been fully allocated no longer need to be in the DLL
for this sizeclass.
* Change to a cycle non-empty list
This change reduces the branching in the case of finding a new free
list. Using a non-empty cyclic list enables branch free add, and a
single branch in remove to detect the empty case.
* Update differences
* Rename first allocation
Use needs initialisation as makes more sense for other scenarios.
* Use a ptrdiff to help with zero init.
* Make GlobalPlaceholder zero init
The GlobalPlaceholder allocator is now a zero init block of memory.
This removes various issues for when things are initialised. It is made read-only
to we detect write to it on some platforms.
If the external thread statics are used, then
we don't need to include some C++ runtime
concepts. This refactoring moves some global initialization under
conditional compilation.
On platforms that do not support aligned mmap/VirtualAlloc,
we need to produce heavily aligned blocks to guarantee we can meet
all possible alignment requests.
This commit grabs a block much larger than requested, and then produces
"offcuts" before and after the block of smaller/same "large_classes". This
enables one mmap/virtual alloc request to services many other requests
for aligned memory.
The PAL API previously allowed for returning more memory than asked for.
This was when the PAL performed the alignment work, now this is done in
large alloc, so removing from the PAL.
* add rust support
* move aligned_size to sizeclass.h
* add static qualifier
* adjust CMakeLists.txt, may broke CI tests
* fix msvc's complaining on c++17
* use SNMALLOC_FAST_PATH as the decorator of aligned_size
* adapt new alignment algorithm and add related test
Co-authored-by: mjp41 <mattpark@microsoft.com>
* fix test cases for msvc
* add extra test for size == 0
* treat memory block of same sizeclass as the same
* fix formatting problem
* remove extra declarations
Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
If the test happens as uintptr_t on CHERI, then we attempt to construct
a capability and use a capability-based test rather than an
integer-based one, and things go south.
posix_memalign requires that the alignment parameter be a multiple of
sizeof(uintptr_t), but the test begins with alignments as small as
sizeof(size_t). While those are very likely the same value out in the
wild right now, they're not on CHERI.
Begin the test loop at sizeof(uintptr_t) and add a test that a request
for a reasonable amount of memory but with an alignment of
sizeof(uintptr_t)/2 fails with EINVAL.
If some tests on Windows are co-scheduled, then they run out of commit
space and crash. For example, in func-memory,
test_external_pointer_large can cause the small CI machines to run out
of commit space on Windows.
Made the API so that get always returns an initialised Alloc*. Added
new fast path that doesn't perform checking, but can lead to very slow
behaviour if called and reused.
This is useful as codegen is nicer if we use size_t, but the semantics
is uint8_t, and is stored as that in many places in the metadata.
Ultimately should introduce a wrapper to check this invariant.
