In computing, achieving high performance is an ongoing challenge, especially as
applications handle increasingly complex workloads. Memory management is a key factor
in performance, where efficient use of resources is essential. Translation Lookaside
Buffers (TLBs) are crucial in this context, speeding up memory access by caching recent
memory address translations. A TLB, a specialised cache in the memory management unit (MMU),
reduces the time required to convert virtual addresses to physical ones. When a program accesses
data in memory, the MMU first checks the TLB for a matching entry, avoiding the slower process of
consulting page tables. However, as applications grow more complex, the fixed size of
applications handle increasingly memory intensive workloads. Memory management is a key factor
in reducing the time it takes to accessa memory region. Translation Lookaside Buffers (TLBs)
are crucial in speeding up memory access by caching recent
memory address translations. A TLB is a specialised cache in the memory management unit (MMU),
It reduces the time required to convert virtual addresses to physical addresses. When a program accesses
data in memory, the MMU first checks the TLB for a matching entry and avoids the slow process of
accessing the page tables situated in memory. However, as applications grow more complex, the fixed size of
TLBs often cannot keep up, leading to more TLB misses and performance slowdowns~\cite{mittal_survey_2017}.
To tackle this issue, researchers have explored new solutions, including the use of
huge pages~\cite{panwar_hawkeye_2019}.
Huge pages, also known as large pages, allow for the allocation of memory in significantly larger chunks
compared to traditional small pages. By reducing the number of TLB entries needed to access a given amount
of memory, huge pages offer a potential avenue for optimising TLBs are used by reducing the number
of memory, huge pages offer a potential avenue for optimising TLBs which are used by reducing the number
of entries needed to map large memory regions. This not only decreases the frequency of
TLB misses but also lowers the overhead associated with address translation. By minimising
these bottlenecks, huge pages can improve system performance in aspects such as speeding
@@ -350,17 +350,17 @@ workloads that rely heavily on large datasets.
Simultaneously, advancements in hardware-level security, such as the Capability Hardware Enhanced RISC Instructions (CHERI)
~\cite{woodruff_cheri_2014} architecture presents additional opportunities for performance enhancement. CHERI's capability-based addressing approach not
only strengthens system security by tightly controlling memory access but also opens avenues for optimising memory management
operations. By integrating CHERI’s compressed encoded bounds~\cite{woodruff_cheri_2019} with the use of huge pages, We have shown it is possible to track and manage
large, physically contiguous memory blocks without requiring numerous TLB entries. This combination reduces TLB pressure by minimising the number of
operations. By integrating CHERI's compressed encoded bounds~\cite{woodruff_cheri_2019} with the use of huge pages, We have shown it is possible to track and manage
large, physically contiguous memory blocks without requiring numerous TLB entries. This combination reduces the TLB pressure by minimising the number of
entries required to map extensive memory regions, thereby decreasing TLB misses and improving address translation performance.
Furthermore, it accelerates memory-intensive tasks by reducing the overhead associated with managing non-contiguous
Furthermore, we introduce FAT which accelerates memory-intensive tasks by reducing the overhead associated with managing non-contiguous
memory allocations. The contributions for the following paper are as follows:
\begin{itemize}
\item\textbf{FAT Addresses Translations}: Introduces FAT that include memory bounds, allowing
efficient tracking and management of physically contiguous memory regions (Section ~\ref{sec:FatPointerTranslations}).
\item\textbf{CHERI’s Capability-based Optimization}: Demonstrates how CHERI's architecture can be
used to optimize memory allocation by encoding memory bounds directly within pointers, reducing TLB reliance
\item\textbf{CHERI’s Capability-based Optimisation}: Demonstrates how CHERI's architecture can be
used to optimise memory allocation by encoding memory bounds directly within pointers, reducing TLB reliance
(Section ~\ref{sec:128bitCompressedBounds}).
\item\textbf{Memory Allocation Algorithms}: Provides an algorithms for allocating, freeing
@@ -368,10 +368,10 @@ memory allocations. The contributions for the following paper are as follows:
(Section ~\ref{sec:MemoryAllocator}).
\end{itemize}
Through comprehensive evaluation, including micro and macro benchmarks, we demonstrate the allocator’s ability
to reduce TLB misses by up to 90\%, yielding significant improvements in wall clock runtimes for memory-intensive
applications. While its impact on larger, computation-heavy workloads is less pronounced,
the proposed allocator shows strong potential for advancing memory management in scenarios requiring
Through comprehensive evaluation including micro and macro benchmarks, we demonstrate the allocator's ability
to reduce TLB misses by up to 90\% which yields in significant improvements in wall clock runtimes for memory-intensive
applications. While its impact on larger and computation-heavy workloads is less pronounced.
The proposed allocator shows strong potential for advancing memory management in scenarios requiring
high memory throughput by reducing the address translation overhead.
\section{Related work}
@@ -385,26 +385,26 @@ high memory throughput by reducing the address translation overhead.
% and end of the segment. Any virtual address within this region
% can be translated by adding the fixed offset between the virtual
% and physical address.
Increasing TLB reach\cite{TLBReach} can be achieved by using larger page sizes, such as huge pages~\cite{panwar_hawkeye_2019}, which are common in modern computer systems.
The x86-64 architecture supports huge pages of 2 MB and 1 GB, backed by OS mechanisms like Transparent Huge Pages (THP)~\cite{THP}
and HugeTLBFS in Linux. However, available page sizes in x86-64 are limited, leading to internal fragmentation issues.
Increasing TLB reach\cite{TLBReach} can be achieved by using larger page sizes such as huge pages~\cite{panwar_hawkeye_2019} which are common in modern computer systems.
The x86-64 architecture supports huge pages of 2 MB and 1 GB which are backed by OS mechanisms like Transparent Huge Pages (THP)~\cite{THP}
. However, available page sizes in x86-64 are limited, leading to internal fragmentation issues.
with architectural capabilities to enable fine-grained memory protection
and highly scalable software compartmentalization. It is a hybrid capability
and highly scalable software compartmentalisation. It is a hybrid capability
architecture that can combine capabilities with conventional MMU (Memory Management Unit)
based systems. The contributions of CHERI include ISA changes to introduce architectural
capabilities; a new microarchitecture that demonstrates capabilities can be implemented efficiently in hardware,
with support for efficient tagged memory to protect capabilities and compress them to reduce memory overhead;
a newly designed software construction model that uses capabilities to provide fine-grained memory protection and scalable
software compartmentalization; language and compiler extensions for using capabilities with C and C++; and OS extensions to
software compartmentalisation; language and compiler extensions for using capabilities with C and C++; and OS extensions to
support fine-grained memory protection (including spatial, referential, and non-stack temporal memory safety) and abstraction extensions
for scalable software compartmentalization.
for scalable software compartmentalisation.
\subsection{CHERI CC}
CHERI Concentrate: Practical Compressed Capabilities\cite{woodruff_cheri_2019} introduces a compression scheme for CHERI, aims to address the performance and compatibility challenges associated with
capability pointers. Capability pointers enhance memory safety by embedding bounds and permissions directly
within pointers, but traditional implementations double their size—leading to increased memory usage. CHERI CC
proposes a compression strategy that preserves security while reducing size and inefficiencies. Key contributions include a floating-point
bound encoding technique with an internal exponent mechanism that offers greater precision for smaller objects and optimized space usage for larger ones.
bound encoding technique with an internal exponent mechanism that offers greater precision for smaller objects and optimised space usage for larger ones.
\section{Fat Address Translations}
\label{sec:FatPointerTranslations}
@@ -659,7 +657,7 @@ on physically contiguous memory.
\label{sec:MemoryAllocator}
This section presents a straightforward memory allocator designed and implemented based on the
principles outlined in FAT (Section ~\ref{sec:FatPointerTranslations}). The allocator consists of three core functions: \textit{InitAlloc},
\textit{malloc}, and \textit{free}. The \textit{InitAlloc} function initializes the memory pool, setting up the necessary
\textit{malloc}, and \textit{free}. The \textit{InitAlloc} function initialises the memory pool, setting up the necessary
data structures and metadata required for efficient memory management. The \textit{malloc} function is
responsible for allocating a contiguous block of memory of a specified size, while the \textit{free}
function deallocates the memory, returning it to the pool for future use.
@@ -740,8 +738,8 @@ efficient memory management but also demonstrate a practical use case of huge pa
\end{algorithmic}
\end{algorithm}
Algorithm \ref{alg:initAlloc} describes the initialization of physically contiguous memory through the use of huge pages,
a mechanism supported by modern architectures to optimize memory management. The algorithm begins by
Algorithm \ref{alg:initAlloc} describes the initialisation of physically contiguous memory through the use of huge pages,
a mechanism supported by modern architectures to optimise memory management. The algorithm begins by
allocating a fixed block of 1 GB of physically contiguous memory. This decision is driven by the
architectural constraints of contemporary systems, particularly ARM-based CPUs, where 1 GB represents
the largest supported page size. By leveraging huge pages, the algorithm reduces the overhead associated
@@ -755,7 +753,7 @@ Jemalloc is the default memory allocator for CHERIBSD~\cite{cheribsd}, to assess
enabled by the FAT allocator. Specifically, we evaluated
the reduction in TLB walks and misses and its impact on wall clock runtime.
To comprehensively analyze the proposed allocator, we categorized benchmarks into
To comprehensively analyse the proposed allocator, we categorised benchmarks into
two classes which are micro and macro benchmarks. Micro benchmarks comprise smaller
C programs designed to target specific allocator patterns, enabling us to evaluate
detailed aspects of the allocator's behavior. Macro benchmarks, on the other hand,
@@ -876,26 +874,26 @@ and varying execution contexts.
and emulated multi-threading scenarios using various block sizes and
allocation patterns. It simulates real-world memory usage by partially
deallocating blocks in FIFO order and fully deallocating them in LIFO order.
Results are gathered across configurations to analyze performance variations.
Results are gathered across configurations to analyse performance variations.
\item\texttt{MemAccess}: This benchmark by Alex Bordei evaluates the performance impact of
memory access patterns by constructing and traversing a doubly
linked list with varying working set sizes. It supports sequential or
randomized structures, optional node operations, and multithreaded
randomised structures, optional node operations, and multithreaded
traversal using pthreads. The program dynamically allocates memory and systematically
doubles the working set size to analyze memory hierarchy behavior.
doubles the working set size to analyse memory hierarchy behavior.
\end{itemize}
\subsubsection{Macro benchmark}
\label{sec:Macro}
\begin{itemize}
\item\texttt{Kmeans}: Kmeans implements a parallelized K-means clustering algorithm that
\item\texttt{Kmeans}: Kmeans implements a parallelised K-means clustering algorithm that
assigns data points to clusters based on proximity to centroids,
iteratively updating them until convergence. The computation is
distributed across threads using the pthread library, dynamically
assigning tasks to optimize performance. Parameters like data size
assigning tasks to optimise performance. Parameters like data size
and clusters are configurable, and the program ensures efficient
memory management and synchronization.
memory management and synchronisation.
\item\texttt{Richards}: Richards is a task scheduling benchmark that simulates a
multitasking environment with tasks of varying types and priorities,
communicating through queued packets. The schedule function manages
@@ -919,7 +917,7 @@ This extension is described by Holt and Singh ~\cite{holt1995}.
\end{figure*}
The graph (Figure \ref{fig:bargraph}) highlights the performance comparison between the modified memory allocator and
Jemalloc, the default memory allocator. The FAT memory allocator, specifically optimized
Jemalloc, the default memory allocator. The FAT memory allocator, specifically optimised
for use with huge pages, demonstrates a clear advantage in scenarios where memory allocation
patterns benefit from its design. The results align with expectations, showcasing the impact
of its capability to handle memory more efficiently by leveraging huge pages.
@@ -953,12 +951,12 @@ The microbenchmarks, which are crafted to emphasize memory read operations, high
allocator's strengths. These tests simulate frequent and intensive memory access patterns,
where the reduction in TLB misses directly translate into measurable performance gains.
On average, the FAT allocator achieves a 50\% reduction in wall clock runtimes for
these workloads, underscoring its ability to optimize high-throughput memory operations.
these workloads, underscoring its ability to optimise high-throughput memory operations.
On the other hand, macro benchmarks, which represent larger and more complex real-world applications ,
exhibit minimal differences in wall clock runtimes when using the FAT allocator.
This outcome is expected, as macro benchmarks typically involve a broader range of operations
beyond memory allocation, diluting the impact of the allocator's optimizations. Additionally,
beyond memory allocation, diluting the impact of the allocator's optimisations. Additionally,
the benefits of huge pages may be less pronounced for these workloads, as they are often
bottlenecked by factors such as computation or I/O rather than memory translation overhead.
@@ -970,13 +968,13 @@ bottlenecked by factors such as computation or I/O rather than memory translatio
The K-means algorithm was executed with varying cluster sizes to evaluate the performance difference
between the FAT allocator and Jemalloc as the workload scales. This analysis
aims to understand how the allocator's optimizations, particularly its ability to manage memory
aims to understand how the allocator's optimisations, particularly its ability to manage memory
more efficiently with huge pages, impact performance under different workload conditions.
For most cluster sizes tested, the percentage difference in performance remained relatively
consistent. This indicates that the allocator's efficiency scales predictably with increasing
workload sizes, suggesting a stable and uniform benefit across different configurations. The
consistent performance gain is likely due to the allocator's ability to minimize TLB misses
consistent performance gain is likely due to the allocator's ability to minimise TLB misses
and efficiently manage memory allocations for the centroid and data point structures used in
the K-means algorithm.
@@ -986,7 +984,7 @@ temporarily offsets the advantages of the FAT allocator. For example, the memory
might interact with system-level caching mechanisms or TLB behavior differently, leading to an
unexpected change in performance. Additionally, the increased complexity of managing a higher
number of clusters might introduce computational overhead that overshadows the memory allocator's
optimizations.
optimisations.
% This observation highlights the importance of testing across a range of workload sizes and
% configurations to uncover edge cases or specific scenarios where performance deviates from the
@@ -999,7 +997,7 @@ optimizations.
\label{sec:Analysis}
The FAT memory allocator demonstrates significant potential for enhancing
memory management in systems that benefit from huge page optimizations. Its design
memory management in systems that benefit from huge page optimisations. Its design
effectively reduces TLB misses, achieving up to 90\% fewer data TLB walks, L2 TLB reads,
and TLB refills compared to Jemalloc. These improvements lead to noticeable performance
gains, especially in micro benchmarks, where the allocator reduces wall clock runtimes
@@ -1041,7 +1039,7 @@ This paper addresses the growing disparity between application workloads and the
To mitigate this gap, we proposed leveraging physically contiguous memory with CHERI bounds to reduce TLB walks.
We designed a memory allocator that uses huge pages with the CHERI CC scheme to track allocations within the
allocated huge page. This approach reduces the number of TLB entries needed while using bounds
to minimize fragmentation.
to minimise fragmentation.
% Additionally,
% the report explores advancements in system security, particularly through the Capability Hardware Enhanced RISC Instructions (CHERI)
% architecture. CHERI's capability-based addressing enhances system security by associating capabilities with memory pointers,
@@ -21,6 +21,18 @@ abstract = {While superpages are an efficient solution to increase TLB reach, st
file={Navarro - Practical, transparent operating system support fo.pdf:/Users/akilan/Zotero/storage/R9MSCWQX/Navarro - Practical, transparent operating system support fo.pdf:application/pdf},
}
@INPROCEEDINGS{TLBReach,
author={Pham, Binh and Bhattacharjee, Abhishek and Eckert, Yasuko and Loh, Gabriel H.},
booktitle={2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA)},
title={Increasing TLB reach by exploiting clustering in page translations},
- [ ] Check if Rob mentioned paraphrasing setence 1.
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
