This PR provides a templated parameter to the allocation routines. This can be used to add special behaviour in
both the successful allocation behaviour, and in the failing
to allocate cases.
The intent of this is two enable two future features
* set_new_handler - so that the failure case doesn't just set ENOMEM, and return nullptr. But can handle both the Windows and C++ versions of (_)set_new_handler.
* The success handler can be used to add checking, zeroing and in the future storing precise size information in metadata for each allocation.
* Fix off by one in deciding where to switch from small to large.
* Improve codegen on Windows by making it simpler for compiler to remove a redundant branch
Some CI pipelines were occasionally timing out due to the func-memory
test taking too long. This change reduces the number of iterations
from 100 to 50 for each test run, which should help avoid timeouts
while still providing sufficient coverage.
It also adds some debug output to indicate the time taken for each test.
Commit 5680cf6dc6e2 ("jemalloc: don't expose 3.0 compat symbols") in
FreeBSD src removes the default symbols for allocm() and friends, so
this test no longer links. Compatibility is stil provided for programs
linking against FBSD_1.3, but here it seems easier to simply stop
referencing them.
This PR updates to aspects of the CI to fix blocked PRs:
It moves the Bazel CI into its own set of actions, so failure can be isolated from other tasks.
Updates the clang-cl version used in CI for self-vendored build as it was failing with the default.
* handle reentrancy during initialization
* use finialization list if possible
* Add test for reentrancy of C++ destructors and allocation
* Add test for reentrancy of setspecific
* Add new mode for directly calling __cxa_thread_atexit_impl
---------
Co-authored-by: Schrodinger ZHU Yifan <yifanzhu@rochester.edu>
The pool sort routine is used by Verona's systematic testing. There was a bug introduced in #612 that could corrupt the list when a sort occured.
This is not used by snmalloc and the test coverage was insufficient to expose the bug.
This PR fixes the bug, and improves test coverage.
When using snmalloc as header-only library in a project that globally defines `WIN32_LEAN_AND_MEAN`, there is a compile error for macro redefinition.
Guard the pal_windows.h define.
* Lift checking for init to ThreadAlloc
The check init code was tightly integrated into LocalAllocator. This commit pull that code out into ThreadAlloc, and passes a template parameter into the remaining LocalAllocator to perform the relevant TLS manipulations. This removes some of the awkward layering around register_clean_up.
* Reduce size of test due to failures.
Fully disable lotsofthreads test
Need to investigate if the test is unreliable, or we have actually
regressed perf. A quick mimalloc-bench didn't show any regressions.
* Simplify message queue initialisation
This introduces one additional branch on when processing a batch of messages, but it is likely to only be hit when a lot of messages are processed.
* Patch Domestication test.
* Refactor CoreAlloc/LocalAlloc
This combines the notion of CoreAlloc, LocalAlloc and LocalCache into a single class. Previously, these were separated so that a more complex structure would be stored directly in the TLS. This however, proved to be bad for compatibility if the allocator is part of the libc implementation.
This commit collapses all the stages of the allocator into a single class. This simplifies the sequencing and overall is a nice reduction in complexity.
* Re-enable lots of threads test.
* Reenable concept using alternative lazy checking for concepts.
* Self code review
This changes the shape of check_bounds to take a continuation to call if the bounds check succeeds. This is designed to allow for easily wrapping existing code with a bounds check, e.g.
```
void* memcpy(void* dest, const void* src, size_t n) {
return check_bounds(dest, n, [&] {
return memcpy_impl(dest, src, n);
});
}
```
If a thread forks, while another thread is holding an snmalloc lock, then the allocator could stop working.
This patch attempts to protect against the cases of this. There is one case that is not covered. If a fork occurs during the very first allocation. This can result in the installation of the fork handler racing with the fork, and all bets are off.
* Factor out small sizeclass check
* Update realloc(p,0) semantics
This commit changes the behaviour of realloc(p,0) to be free(p) if p!=nullptr, and malloc(0) if p== nullptr.
* Fix overflow by alignment
* Bug fix: Ensure bytes_free is the total
The bytes freed was not added to the total, but
overrode it. This meant it never fired. This
commit fixes that.
* Factor out explicit Config type
Instead of using snmalloc::Alloc::Config, expose snmalloc::Config, which is then used to derive the allocator type.
* Move globalalloc to front end.
* Remove unneed template parameter from global snmalloc functions.
* Remove SNMALLOC_PASS_THROUGH
VeronaRT now has an abstraction layer which can easily replace the allocator.
Having such a complex integration still in snmalloc does not make sense.
* Take some global functions off of local alloc.
* Drop comparison overloads on atomic Capptr.
Performing a comparison on two atomic ptr is a complex operation, and should not be implicit. The memory model order and such things needs to be considered by the caller.
* Remove function_ref and use templates
The implementation prefers to use templates over the function_ref. This now only exists in the Pal for a currently unused feature.
* Removing function_ref reduces stl needs.
* Remove use of __is_convertible to support older g++
* Inline function that is only used once.
* Remove unused function
* Restrict ThreadAlloc usage to globalalloc
This commit introduces various inline functions on snmalloc:: that perform allocation/deallocation using the thread local allocator.
They remove all usage from a particular test.
* Move cheri checks to own file.
* Refactor is_owned checks.
* Move alloc_size and check_size to globalalloc.
* Minor simplification of dealloc path
* Fix up is_owned to take a config
* Improve usage of scoped allocator.
* Handle Config_ in globalalloc.
* Add stress test benchmark
Co-authored-by: Alexander Nadeau <wareya@gmail.com>
* Add defensive code against spurious wakeup
This commit checks that wait_on_address has not returned spuriously.
* pal: spurious wake up.
The code in the Pal for wake on address was incorrectly assuming the operation returning success meant it had actually changed. The specification allows for spurious wake ups.
This change makes the Pals recheck for a change.
---------
Co-authored-by: Alexander Nadeau <wareya@gmail.com>
* Handle platforms that have `_GLIBCXX_ASSERTIONS`, which require the stdlib++ to be included.
* Stop override of memcpy with FORTIFY_SOURCE enabled
* Add to .gitignore
* Add data for graphs in release notes.
* Minor tidy on CMake.
The Pagemap can potentially be accessed where there is no direct
knowledge of the location. On platforms with lazy commit this can
be handled by the OS faulting in a zero page. Here we modify the
Windows PAL to add an exception handler that will commit pages
in the read only range, i.e. the page map. We still require explicit
notify_using to be able to modify the pagemap.
When processing a remote batch, the system will process every single message that was available at the start of processing.
This can lead to a long pause time if there have been a considerable number of frees to this thread.
This commit introduces a new mechanism to only process messages up to a limit of 1MiB. The limit is configurable using CMake.
Choosing too small a limit can cause freeing to never catch up with the incoming messages.
