Define various parts of random that can be used to make the layout of
memory more random. Thread this through the allocator.
Expose the concept as part of the Pal. Subsequent commits will expose
that on different platforms.
* Make LowMemoryNotification object allocated
This makes a separate allocation for the callback object. This makes
it easier for different callbacks to be used.
* Add reserve_with_leftover to address_space
The address_space now supports reserving for non-power of 2 allocations
and the space that is used for rounding up is retained by the
address_space. This means that we can more tightly pack the allocators
internal objects.
Summary of changes:
- Add a new PAL that doesn't allocate memory, which can be used with a
memory provider that is pre-initialised with a range of memory.
- Add a `NoAllocation` PAL property so that the methods on a PAL that
doesn't support dynamically reserving address space will never be
called and therefore don't need to be implemented.
- Slightly refactor the memory provider class so that it has a narrower
interface with LargeAlloc and is easier to proxy.
- Allow the address space manager and the memory provider to be
initialised with a range of memory.
This may eventually also remove the need for (or, at least, simplify)
the Open Enclave PAL.
This commit also ends up with a few other cleanups:
- The `malloc_useable_size` CMake test that checks whether the
parameter is const qualified was failing on FreeBSD where this
function is declared in `malloc_np.h` but where including
`malloc.h` raises an error. This should now be more robust.
- The BSD aligned PAL inherited from the BSD PAL, which does not
expose aligned allocation. This meant that it exposed both the
aligned and non-aligned allocation interfaces and so happily
accepted incorrect `constexpr` if blocks that expected one or
the other but accidentally required both to exist. The unaligned
function is now deleted so the same failures that appear in CI should
appear locally for anyone using this PAL.
* Sparc support proposal.
* Tweaks from feedback.
Shift to integer/pointers cast like other archs.
Assembly reworks.
Note apparently reading the register once is sufficient to provoke
a pause in the cpu adding a clobber tough.
- Close to OpenBSD as there is no malloc*size api nor arbritrary
alignment support.
- Like FreeBSD, MAP_NORESERVE never had been implemented even tough
still present in the header but not mentioned in the man page,
FreeBSD has reserved the value for another later usage seems
DragonFly has just out of sync header.
This will not be used unless the C++ standard version is raised to 20. As
concepts and C++20 more generally are quite new, this does not do so.
Nevertheless, the use of concepts can improve the local development experience
as type mismatches are discovered earlier (at template invocation rather than
only during expansion).
The POSIX PAL and in some configurations the Windows PAL overallocate
address space. If address space has become exhausted then this can lead
to issues, where the PAL fails, even though there is enough aligned
address space to satisfy the underlying request.
This commit tries increasingly smaller overallocation sizes in an
attempt to succeed in more cases.
* Reduce code duplication.
The Apple and Haiku PALs both had *almost* identical code to the POSIX
PAL, differing only in some small argument variables. This is fragile
and easy to accidentally get out of sync. For example, the changes to
`reserve_at_least` involved copying identical code into multiple PALs
and it's easy to accidentally miss one.
This change introduces two optional fields on POSIX-derived PALs:
- `default_mmap_flags` allows a PAL to provide additional `MAP_*`
flags to all `mmap` calls.
- `anonymous_memory_fd` allows the PAL to override the default file
descriptor used for memory mappings.
If a PAL does not provide these, default values are used.
Fixes#219
This change brings in a new approach to managing address space.
It wraps the Pal with a power of two reservation system, that
guarantees all returned blocks are naturally aligned to their size. It
either lets the Pal perform aligned requests, or over allocates and
splits into power of two blocks.
This changes the implementation of the open enclacve pal to support
aligned allocations. This reduces the amount of memory required for
the initial reservation as the large allocator doesn't have to
overallocate to get alignment.
PALOpenEnclave object is lazily constructed. I couldn't
figure out a straight-forward way to pass the heap bounds to
the constructor of PALOpenEnclave object.
As an alternative, store the bounds in inline static variables of
the PALOpenEnclave class and set them via static setup_initial_range
function.
- two_alloc_types/alloc1.cc
Define oe_allocator_init to forward base, end values to
PALOpenEnclave::setup_inital_range
- two_alloc_types/main.cc
Use oe_allocator_init function to set up heap range.
- fixed_region/fixed_region.cc
Initialize heap range via call to PALOpenEnclave::setup_inital_range.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
The Pal should include address.h, this was masked as other Pals included
it, but are only included for simulating OE scenarios, rather than
the actual build for OE.
* Defensive code for alloc/dealloc during TLS teardown
If an allocation or deallocation occurs during TLS teardown, then it is
possible for a new allocator to be created and then this is leaked. On
the mimalloc-bench mstressN benchmark this was observed leading to a
large memory leak.
This fix, detects if we are in the TLS teardown phase, and if so,
the calls to alloc or dealloc must return the allocator once they have
perform the specific operation.
Uses a separate variable to represent if a thread_local's destructor has
run already. This is used to detect thread teardown to put the
allocator into a special slow path to avoid leaks.
* Added some printing first operation to track progress
* Improve error messages on posix
Flush errors, print assert details, and present stack traces.
* Detect incorrect use of pool.
* Clang format.
* Replace broken LL/SC implementation
LL/SC implementation was broken, this replaces it with
a locking implementation. Changes the API to support LL/SC
for future implementation on ARM.
* Improve TLS teardown.
* Make std::function fully inlined.
* Factor out PALLinux stack trace.
* Add checks for leaking allocators.
* Add release build of Windows Clang
* Remote dealloc refactor.
* Improve remote dealloc
Change remote to count down to 0, so fast path does not need a constant.
Use signed value so that branch does not depend on addition.
* Inline remote_dealloc
The fast path of remote_dealloc is sufficiently compact that it can be
inlined.
* Improve fast path in Slab::alloc
Turn the internal structure into tail calls, to improve fast path.
Should be no algorithmic changes.
* Refactor initialisation to help fast path.
Break lazy initialisation into two functions, so it is easier to codegen
fast paths.
* Minor tidy to statically sized dealloc.
* Refactor semi-slow path for alloc
Make the backup path a bit faster. Only algorithmic change is to delay
checking for first allocation. Otherwise, should be unchanged.
* Test initial operation of a thread
The first operation a new thread takes is special. It results in
allocating an allocator, and swinging it into the TLS. This makes
this a very special path, that is rarely tested. This test generates
a lot of threads to cover the first alloc and dealloc operations.
* Correctly handle reusing get_noncachable
* Fix large alloc stats
Large alloc stats aren't necessarily balanced on a thread, this changes
to tracking individual pushs and pops, rather than the net effect
(with an unsigned value).
* Fix TLS init on large alloc path
* Add Bump ptrs to allocator
Each allocator has a bump ptr for each size class. This is no longer
slab local.
Slabs that haven't been fully allocated no longer need to be in the DLL
for this sizeclass.
* Change to a cycle non-empty list
This change reduces the branching in the case of finding a new free
list. Using a non-empty cyclic list enables branch free add, and a
single branch in remove to detect the empty case.
* Update differences
* Rename first allocation
Use needs initialisation as makes more sense for other scenarios.
* Use a ptrdiff to help with zero init.
* Make GlobalPlaceholder zero init
The GlobalPlaceholder allocator is now a zero init block of memory.
This removes various issues for when things are initialised. It is made read-only
to we detect write to it on some platforms.
