Commit Graph

943 Commits

Author SHA1 Message Date
Matthew Parkinson
4f2d3ebf33 Enable sanitizers in CI. 2021-12-15 14:45:53 +00:00
Matthew Parkinson
0fd5c37563 Change MPSCQ to use acquire
TSAN complained that there was a race that after some thoughts appears
to be due to this exchange needing to be an `acquire`.

I still wonder if the data dependence that is threaded through the
exchange induces enough order.
2021-12-15 14:45:53 +00:00
Matthew Parkinson
7768765fe8 Fixed locking around notification
Deduplication locking had test_and_set incorrect direction.
2021-12-15 14:45:53 +00:00
Matthew Parkinson
f398545154 Fix memalign usage
- Alignment should be above sizeof(void*)
- Don't call with very large sizes
2021-12-15 14:45:53 +00:00
Schrodinger ZHU Yifan
37d2ac42af use ::write and ::fsync in error path (#443)
* use ::write and ::fsync in error path

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* mark return value unused

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* adjust fsync positions

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-12-15 14:01:34 +00:00
Matthew Parkinson
d0df2d028a Fix CI branch name. (#442) 2021-12-13 14:36:12 +00:00
David CARLIER
360efa2123 export netbsd's reallocarr proposal. (#433)
* export netbsd's reallocarr proposal.

acts subtly differently from reallocarray, returns an error code
and first argument as receiver.

* not export by default

* ci tests

* apply suggestions

* doc addition

* Apply suggestions from code review

Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2021-12-02 14:49:32 +00:00
Matthew Parkinson
7f3642e05c Change external thread_alloc example. (#424)
On Open Enclave having the `local_alloc` directly in thread-local
storage was causing a crash.  This changes the `local_alloc` to be
indirected, and thus puts less pressure on the thread-local storage.

The test also has deals with how to allocate before a thread-local
storage has been established.
2021-12-02 10:39:44 +00:00
Schrodinger ZHU Yifan
71d5bb8756 add SYS_getrandom fallback for posix PAL (#431)
* add SYS_getrandom fallback for posix PAL

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* address CR

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* guard and comments

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* more fallback behavior

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* fix random device fd

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* special handling for EAGAIN

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-12-02 10:38:50 +00:00
Matthew Parkinson
9329db102f Remove Win2016 as Github Action support ending. 2021-12-02 09:58:02 +00:00
Matthew Parkinson
894b0314c9 Add timeout to ci tests.
Windows can hang due to assert failures in CI.  Add a timeout to get
some information of what is happening.
2021-12-02 09:58:02 +00:00
Schrodinger ZHU Yifan
9b548ed3a2 Add ownership checkings for FlagLock under debug (#432)
* add ownership checkings for FlagLock under debug

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* fix owner reset ordering

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* addresss CR

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* fix include and constexpr problem

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* use thread_local variable as thread identity

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* change default form for initialization

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* address CR

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* fix typo and format

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* add more assertions

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* adjust flag lock and comments

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* address CR

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-12-02 09:18:24 +00:00
David Carlier
b1da339b3e build fix proposal for GCC 11.x (spotted with 11.2.0).
build error due to access none attribute on the pthread_setspecific's
value argumenti (glibc).
2021-12-01 10:05:52 +00:00
Matthew Parkinson
5fd3288997 Modify heuristic for adding new slabs. (#429)
If there is only one slab remaining, then we probabalisticly allocator a
new one. If a slab is barely in use, then this could cause us to
effectively double the number of slabs in use.

This commit checks if the remaining slab has enough remaining elements
to provide randomisation.
2021-11-25 13:43:50 +00:00
Matthew Parkinson
c299826f58 Improve codegen for checks.
Use fail fast in release to avoid stack frame for error reporting.

Scope check_client macro.
2021-11-25 08:22:28 +00:00
Matthew Parkinson
a8ef963ed7 Small comment 2021-11-25 08:22:28 +00:00
Matthew Parkinson
d725beca90 Minor improvement to codegen for remaining_bytes 2021-11-25 08:22:28 +00:00
David CARLIER
9332557bb0 memory unit test android build fix. (#427) 2021-11-24 12:12:47 +00:00
David CARLIER
e24130137d solaris systems build fix (#425) 2021-11-22 09:47:30 +00:00
Schrodinger ZHU Yifan
8e5514bd5a clean up unused usages (#421)
* clean up unused usages

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* remove names for arg pack

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* fix namespace in setup.h

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* format

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* set UNUSED as fast path

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-11-19 13:23:39 +00:00
David Carlier
f731bc169b build fix for some 3rd party oses. 2021-11-18 09:08:17 +00:00
Schrodinger ZHU Yifan
faa80037bb put likely/unlikely in scope (#420)
* put likely/unlikely in scope

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* make clang-format happy

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-11-17 16:05:52 +00:00
Schrodinger ZHU Yifan
cd0311b26f Nits for rust release (#419)
* adjust gitignore

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* also add prefix for rust objects

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* export statistics api for rust

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>

* conform clang-format-9

Signed-off-by: SchrodingerZhu <i@zhuyi.fan>
2021-11-17 16:02:47 +00:00
Matthew Parkinson
dc542cdc9d Various fixes for OE (#418)
* Add default for getting chunk allocator state

Makes the API same between the two configurations.

* Reduce address space usage for Open Enclave

* Fix OE Pal concept

* Add support for Pal not to provide time.

The lazy return of pages to the OS uses a simple time
based heuristic.  This enables a PAL to not support time,
and return the memory to a central pool immediately.

* Update src/backend/backend.h

Co-authored-by: Amaury Chamayou <amaury@xargs.fr>

Co-authored-by: Amaury Chamayou <amaury@xargs.fr>
2021-11-17 14:11:46 +00:00
Matthew Parkinson
91919f6014 Update src/backend/pagemap.h
Co-authored-by: Amaury Chamayou <amchamay@microsoft.com>
2021-11-17 09:19:08 +00:00
Matthew Parkinson
3d403aef7f Refactor use of sizeclasses (#415)
The primary aim for this refactor is to use a representation for
sizeclasses that uniformly covers both large and small.  This allows
certain operations such as alloc_size and external_pointer to be
uniformly implemented.

The additional types make clear which kind of sizeclass is in use.

This also tidies up the code for sizeclass based divisible by and
modulus.

It fixes a bug in rust_realloc that didn't correctly determine a realloc
was required for large classes.
2021-11-10 16:35:44 +00:00
Matthew Parkinson
02f36a4b31 Make test fail with more information. 2021-11-03 20:22:50 +00:00
Matthew Parkinson
cf89339762 Move rsize calculation to a slow path. 2021-11-03 20:22:50 +00:00
Matthew Parkinson
d01e2cdf60 Fix tests.
Release tests require CTest to actually be passed the -C option.
2021-11-03 20:22:50 +00:00
Nathaniel Wesley Filardo
7ef1dfdd51 Fix ARM AAL on Morello
The spelling of inline assembler constraints on Morello would be
different, but Jessica Clarke points out that we should just be using
the builtin when available, so do that instead.

Co-authored-by: Jessica Clarke <jrtc27@jrtc27.com>
2021-11-03 18:23:33 +00:00
Matthew Parkinson
dd5c91eb43 Change Remote Cache default size
With the new snmalloc2 changes it seems the larger window is leading to
more fragmentation and harming performance.  Reducing size still
provides good batching, improves memory overhead.
2021-11-02 19:56:50 +00:00
Matthew Parkinson
19de27fdaf Improve fast path of handle remote deallocs
This adds some branch predictor and cache hints to the fast path of
processing remote deallocation. It also removes the batching.
2021-11-02 19:56:50 +00:00
Matthew Parkinson
fd408637a7 Move some structures to cache lines
Prevent some bad sharing of cache lines by aligning some concurrently
accessed strucutures.
2021-11-02 19:56:50 +00:00
Matthew Parkinson
3407347925 Add custom datastructure for local chunk cache
This commit adds a datastructure that provides efficient single-thread
stack behaviour, while allowing other threads to steal the whole
contents.
2021-11-02 19:56:50 +00:00
Matthew Parkinson
72ccb23d02 Add local caching to chunk allocator 2021-10-28 14:28:36 +01:00
Matthew Parkinson
20a114cb62 Add a timer to the PAL
This adds a way to periodically pool the PAL to see if any timers have
expired.  Timers can be used to periodically provide callbacks to the
rest of snmalloc.
2021-10-28 14:28:36 +01:00
Matthew Parkinson
c1062e629e Add Madvise free to Linux layer 2021-10-28 14:28:36 +01:00
Matthew Parkinson
cec015f296 Add FIFO behaviour for unchecked code.
Consuming available slabs in LIFO order makes predicting address reuse harder
but appears to have performance implications.  Condition this on CHECK_CLIENT
and instead use FIFO order on !CHECK_CLIENT builds.
2021-10-22 17:16:14 +01:00
Matthew Parkinson
4987a19fe9 Rename 2021-10-22 17:16:14 +01:00
Matthew Parkinson
5215bffa9e Change malloc test suite for errno
Errno is not required to be 0 on return from malloc,
so don't bother trying to make it 0. Leads to false test failures where
libc calls have not reset it after a failure.
2021-10-21 16:34:17 +01:00
Matthew Parkinson
eb6c1a05c8 Make chunk size at least the page size.
On systems with larger than 16KiB page size, we have chunks
that divide a page.  This seems a little strange, and if we
want to disable the pages backing a chunk, this is not possible.

This change ensures the chunk is always at least a single page.
2021-10-21 11:56:10 +01:00
Matthew Parkinson
4a7cd268f8 Rename slab_allocator to chunk_allocator 2021-10-20 18:38:08 +01:00
Nathaniel Wesley Filardo
342d826310 CheriBSD bugs workarounds
The correct thing to do, of course, is to fix these upstream, but that requires
understanding exactly what's wrong, and that's harder than just not tickling the
bugs.
2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
6ddd11faee CheriBSD/CHERI support
This adds a CHERI AAL and expands the FreeBSD PAL to cover CHERI.  It updates a
comment in ds/address.h now that there is an example architecture that
differentiates uintptr_t and address_t.
2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
6f79ddee31 FreeBSD RISC platforms have fewer address bits 2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
eb0698fc09 CI: Add RISC-V 64 cross-build & qemu-user tests 2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
97a8bd1758 Initial RISC-V AAL 2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
fe43f0bea8 Placate some compiler errors on CHERI 2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
fa4560801a freelist: check_prev expects an address_t
So make signed_prev return one.  This distinction only matters on CHERI, of
course; everywhere else address_t *is* a uintptr_t.
2021-10-20 12:02:08 +01:00
Nathaniel Wesley Filardo
8023a6c906 backend/address_space: drop spurious !StrictProvenance guard
This dates back to the much earlier design that required the use of an authority
map.  Since the current CHERI design does not, go ahead and ask the platform
whenever underlying allocation requests are sufficiently aligned.
2021-10-20 12:02:08 +01:00