This PR updates to aspects of the CI to fix blocked PRs:
It moves the Bazel CI into its own set of actions, so failure can be isolated from other tasks.
Updates the clang-cl version used in CI for self-vendored build as it was failing with the default.
- Split ubuntu and macos CI actions, even though they use very similar steps
- Remove macos-11, keep -12, and add -14
- Have all macos platforms build with and without C++17
- Remove duplicated dependency lines in ubuntu matrix entries; push this down
to the steps
- Ensure that all added ubuntu matrix tuples have non-empty build-type
- Add all jobs to all-checks' "needs:" to ensure we wait for everything
This provide a way to configure snmalloc to provide per object meta-data that is out of band. This can be used to provide different mitigations on top of snmalloc, such as storing memory tags in a compressed form, or provide a miracle pointer like feature.
This also includes a couple of TSAN fixes as it wasn't fully on in CI.
The current version requires clang-format-9. This now getting hard to get.
This commit moves it to the clang-format-15, which is the latest in 22.04.
Also, updates clang-tidy to 15 as well.
All the checks and mitigations have been placed under feature flags.
These can be controlled by defining
SNMALLOC_CHECK_CLIENT_MITIGATIONS
This can take a term that represents the mitigations that should be enabled.
E.g.
-DSNMALLOC_CHECK_CLIENT_MITIGATIONS=nochecks+random_pagemap
The CMake uses this to build numerous versions of the LD_PRELOAD library and
tests to allow individual features to be benchmarked.
Co-authored-by: Nathaniel Wesley Filardo <nfilardo@microsoft.com>
* Move Morello CI to track default release
- Log some details of the build environment
- Remove workarounds overcome by events
* Morello CI: parameterize run queue and boot env
* Morello CI to run as a non-root user
For reasons unrelated to snmalloc, it's become more convenient to engage
in a little white lie, as it were, that the CI jobs are not `root` on
the worker nodes. So I'm testing changes on the cluster orchestration
goo to run the github runner as a non-root user. However, much as with
GitHub's own runners, the runner user is in the `wheel`, and `root` will
have no password, so we can still `su` up to `root` when needed.
Of course, when we are already root, we can `su` to anyone we like,
including `root`, so these changes are compatible with both the old and
new world order and have been tested with both.
This uses VirtualBox instead of xhyve. It might be slower, but should
be more reliable.
Tests run on FreeBSD, NetBSD, and OpenBSD. Only the FreeBSD ones are
passing at the moment, the others will keep running but aren't added as
dependencies for the action used to guard commits.
See src/snmalloc/README.md for an explanation of the layers.
Some other cleanups on the way:
Fine-grained stats support is now gone.
It's been broken for two years, it depends on iostream (which then
causes linker failures with libstdc++) and it's collecting the wrong
stats for the new design. After discussion with @mjp41, it's better to
remove it and introduce new stats support later, rather than keep broken
code in the main branch.
Tracing was controlled with a preprocessor macro, now there's also a
CMake option.
The memcpy implementation is not completely stupid but is almost
certainly not as good as a carefully tuned and optimised one.
Building snmalloc with FreeBSD's libc memcpy + jemalloc and with this,
each 10 times, does not show a statistically significant performance
difference at 95% confidence. The snmalloc version has very slightly
lower median and worst-case times. This is in no way a sensible
benchmark, but it serves as a smoke test for significant performance
regressions.
The CI self-host job now uses the checked memcpy.
This also fixes an off-by-one error in the external bounds. This is
triggered by ninja, so we will see breakage in CI if it is reintroduced.
In debug builds, we provide a verbose error containing the address of
the allocation, the base and bounds of the allocation, and a backtrace.
The backtrace was broken by the CI cleanup moving the BACKTRACE_HEADER
macro into the SNMALLOC_ namespace. This is also fixed.
The test involves hijacking `abort`, which doesn't work everywhere. It
also requires `backtrace` to work in configurations where stack traces
are enabled. This is disabled in QEMU because `backtrace` appears to
crash reliably in QEMU user mode.
For now, in the -checks build configurations, we are hitting a slow path
in the pagemap on accesses so that the pages that are `PROT_NONE` don't
cause crashes. These need to be made read-only, but this requires a PAL
change.
