From 35afa3f1ac323b2d1039de4ce7338d7b5266c99f Mon Sep 17 00:00:00 2001 From: Akilan Date: Fri, 18 Apr 2025 16:00:57 +0100 Subject: [PATCH] added typo fixes --- docs/EuroSys/Paper/paper.tex | 80 ++--- docs/EuroSys/Paper/paper.txt | 655 +++++++++++++++++++++++++++++++++++ docs/EuroSys/Paper/todo.org | 2 - 3 files changed, 695 insertions(+), 42 deletions(-) create mode 100644 docs/EuroSys/Paper/paper.txt diff --git a/docs/EuroSys/Paper/paper.tex b/docs/EuroSys/Paper/paper.tex index f661dac..5996107 100644 --- a/docs/EuroSys/Paper/paper.tex +++ b/docs/EuroSys/Paper/paper.tex @@ -240,10 +240,10 @@ % our allocator can reduce TLB misses by up to 90\%, leading to improvements in wall clock runtimes for memory-intensive applications. The increasing gap between workload memory requirements and the capacity of translation lookaside buffers (TLBs) means TLB misses - are more frequency, costing additional clock cycles, impacting runtime performance. One solution is to use physically contiguous + are more frequent, costing additional clock cycles and impacting runtime performance. One solution is to use physically contiguous memory in conjunction with huge pages. We propose an alternative approach, by exploiting capability-based addressing in the - CHERI architecture. This paper presents a new memory allocator. It associates capabilities with memory pointers. integrating - block-based allocations within huge pages. Our allocator reduces TLB misses by up to 90\%, which leads to reduced runtimes + CHERI architecture. This paper presents a new memory allocator. It associates capabilities with memory pointers. Integrating + block-based allocations within huge pages. Our allocator reduces TLB misses by up to 90\%, which leads to decreasing runtimes for memory-intensive applications. \end{abstract} @@ -333,14 +333,14 @@ Buffers (TLBs) are crucial in this context, speeding up memory access by caching memory address translations. A TLB, a specialised cache in the memory management unit (MMU), reduces the time required to convert virtual addresses to physical ones. When a program accesses data in memory, the MMU first checks the TLB for a matching entry, avoiding the slower process of -consulting page tables. However, as applications grow larger and more complex, the fixed size of +consulting page tables. However, as applications grow more complex, the fixed size of TLBs often cannot keep up, leading to more TLB misses and performance slowdowns~\cite{mittal_survey_2017}. To tackle this issue, researchers have explored new solutions, including the use of huge pages~\cite{panwar_hawkeye_2019}. Huge pages, also known as large pages, allow for the allocation of memory in significantly larger chunks compared to traditional small pages. By reducing the number of TLB entries needed to access a given amount -of memory, huge pages offer a potential avenue for optimising TLBs use by reducing the number +of memory, huge pages offer a potential avenue for optimising TLBs are used by reducing the number of entries needed to map large memory regions. This not only decreases the frequency of TLB misses but also lowers the overhead associated with address translation. By minimising these bottlenecks, huge pages can improve system performance in aspects such as speeding @@ -348,7 +348,7 @@ up memory-intensive applications, reducing latency in data access, and enhancing workloads that rely heavily on large datasets. Simultaneously, advancements in hardware-level security, such as the Capability Hardware Enhanced RISC Instructions (CHERI) -~\cite{woodruff_cheri_2014} architecture, present additional opportunities for performance enhancement. CHERI's capability-based addressing approach not +~\cite{woodruff_cheri_2014} architecture presents additional opportunities for performance enhancement. CHERI's capability-based addressing approach not only strengthens system security by tightly controlling memory access but also opens avenues for optimising memory management operations. By integrating CHERI’s compressed encoded bounds~\cite{woodruff_cheri_2019} with the use of huge pages, We have shown it is possible to track and manage large, physically contiguous memory blocks without requiring numerous TLB entries. This combination reduces TLB pressure by minimising the number of @@ -364,7 +364,7 @@ memory allocations. The contributions for the following paper are as follows: (Section ~\ref{sec:128bitCompressedBounds}). \item \textbf{Memory Allocation Algorithms}: Provides an algorithms for allocating, freeing - physically contiguous memory and integrating huge pages with CHERI's capability-based bounds for enhanced memory management + physically contiguous memory , and integrating huge pages with CHERI's capability-based bounds for enhanced memory management (Section ~\ref{sec:MemoryAllocator}). \end{itemize} @@ -395,7 +395,7 @@ For instance, allocating 1 MB with 4 KB base pages requires 256 PTEs (Page Table half of the memory space. Some architectures offer more page size choices, such as Intel Itanium, which allows different areas of the address space to have their own page sizes. Itanium uses a hash page table to organize huge pages, but without significant changes to the conventional page table, it only helps reduce page walk overheads. -Huge page tunable base page size permits the OS to adjust the base page size, but still faces internal fragmentation problems, +Huge page tunable base page size permits the OS to adjust the base page, but still faces internal fragmentation problems, with huge page recommending a base page size of no more than 16 KB. Shadow Superpage~\cite{Shadow_superpages} introduces a new translation level in the memory controller to merge non-contiguous physical pages into a huge page in a shadow memory space, extending TLB coverage. However, this approach requires all memory traffic to be translated again in the memory controller, @@ -432,13 +432,13 @@ reducing overhead. Range mapping works alongside the paging system by generating TLB misses and still performing TLB lookups for each virtual address translation. Unlike traditional segmentation mechanisms, range mapping activates a range lookaside buffer (RTLB) located with the last level TLB upon a miss. The hardware TLB miss -handler then searches the RTLB for the miss address and, if found, generates a new +handler then searches the RTLB for the missed address and, if found, generates a new TLB entry with the physical address derived from the base virtual address and range offset, along with permission bits. If the RTLB also misses, the system defaults to a standard page walk while a range table walker simultaneously -loads the range into the RTLB in the background, avoiding delays in memory operations. +loads the range into the RTLB in the background, avoiding delays in-memory operations. The RTLB, functioning as a fully associative search structure, ensures -that most last level TLB misses are handled efficiently by range mapping, +that most last-level TLB misses are handled efficiently by range mapping, reducing the need for costly page table walks. \subsection{CHERI} @@ -471,16 +471,16 @@ support fine-grained memory protection (including spatial, referential, and non- for scalable software compartmentalization. \subsection{CHERI CC} -CHERI Concentrate: Practical Compressed Capabilities\cite{woodruff_cheri_2019} introduces a compression scheme for CHERI, aiming to address the performance and compatibility challenges associated with +CHERI Concentrate: Practical Compressed Capabilities\cite{woodruff_cheri_2019} introduces a compression scheme for CHERI, aims to address the performance and compatibility challenges associated with capability pointers. Capability pointers enhance memory safety by embedding bounds and permissions directly within pointers, but traditional implementations double their size—leading to increased memory usage. CHERI CC proposes a compression strategy that preserves security while reducing size and inefficiencies. Key contributions include a floating-point -bounds encoding technique with an internal exponent mechanism that offers greater precision for smaller objects and optimized space usage for larger ones. +bound encoding technique with an internal exponent mechanism that offers greater precision for smaller objects and optimized space usage for larger ones. \section{Fat Address Translations} \label{sec:FatPointerTranslations} Fat Address Translations (FAT) uses the CHERI architecture to -bring about block based allocations in physically contiguous memory. +bring about block-based allocations in physically contiguous memory. FAT leverages techniques like FlexPointer~\cite{chen_flexpointer_2023} and RMM~\cite{karakostas_redundant_2015} to reduce pressure on the TLB. A key component in this implementation is the use of range addresses with CHERI CC~\cite{woodruff_cheri_2019}. @@ -549,15 +549,15 @@ bounds within the pointers and as shown in the figure \ref{fig:HighOverviewArchi % tracking of memory ranges on a pointer level. A memory range in FAT has two points to track memory in physical contiguous space which is the top and bottom. These two points are two virtual addresses and the range consists of -addresses which lie within this and refers to addresses allocated by invoking \textit{malloc}. -In FAT memory ranges are established using +addresses that lie within this and refers to addresses allocated by invoking \textit{malloc}. +FAT memory ranges are established using bounds encoded within the pointer, adhering to CHERI CC~\cite{woodruff_cheri_2019}. % as referred in section ~\ref{sec:128bitCompressedBounds}. Figure \ref{fig:RangeOfMemory} illustrates a straightforward use-case in which the dark pink line represents a single, large contiguous memory area, or huge page. Within this huge page, the orange and blue lines indicate two separate memory allocations equivalent to invoking \textit{malloc} twice to allocate memory in distinct regions. -This scenario simulates a block-based memory allocator operating within the confines of the huge page. +This scenario simulates a block-based memory allocator operating within the confines of a huge page. The allocations use the bounds encoded in the FAT, ensuring tracking of the allocated memory regions. By using the CHERI bounds, this method maintains the contiguity of the allocated blocks within the huge page. @@ -582,7 +582,7 @@ By using the CHERI bounds, this method maintains the contiguity of the allocated We use CHERI CC~\cite{woodruff_cheri_2019} to track regions of memory in physically contiguous space. CHERI CC consists of compressed bounds that represent a 128-bit pointer within a 64-bit virtual address system. Our approach uses the work of CHERI CC by using a single cycle to decode bounds from the -capability register and the bounds which are decoded are repurposed for tracking memory which is eagerly +capability register and the bounds that are decoded are repurposed for tracking memory which is eagerly allocated. % Our approach utilizes a single-cycle encoding and decoding mechanism for efficiency. While CHERI % CC was originally designed for memory protection, it can also be repurposed for tracking memory regions, @@ -623,8 +623,8 @@ This approach offers a more flexible alternative to the traditional fixed-size T % To build up based on Section ~\ref{sec:RangeMemory} and ~\ref{sec:128bitCompressedBounds}. We are able to pre-allocate memory using huge pages and are able to mark smaller allocations using ranges by storing them as bounds within the pointer. Each of these memory ranges can be -called as a block. Since we have numerous blocks inside a huge page we allow block based -memory patters within physically contiguous memory. +called a block. Since we have numerous blocks inside a huge page we allow block-based +memory patterns within physically contiguous memory. % As demonstrated with the allocator % implementation in section ~\ref{sec:MemoryAllocator}. @@ -649,16 +649,16 @@ memory patters within physically contiguous memory. %and reduces TLB overhead by reducing TLB walks. Figure \ref{fig:HugePages} illustrates a use-case of huge pages where the green -line represents a sample access to read within a contigous -space of physical memory. The dotted lines represents the +line represents sample access to read within a contiguous +space of physical memory. The dotted lines represent the bounds for that particular pointer access. Using bounds -stored on the pointer a block based pattern can be replicated -on physically contigous memory. +stored on the pointer a block-based pattern can be replicated +on physically contiguous memory. \section{Memory allocator design} \label{sec:MemoryAllocator} This section presents a straightforward memory allocator designed and implemented based on the -principles outlined FAT (Section ~\ref{sec:FatPointerTranslations}). The allocator consists of three core functions: \textit{InitAlloc}, +principles outlined in FAT (Section ~\ref{sec:FatPointerTranslations}). The allocator consists of three core functions: \textit{InitAlloc}, \textit{malloc}, and \textit{free}. The \textit{InitAlloc} function initializes the memory pool, setting up the necessary data structures and metadata required for efficient memory management. The \textit{malloc} function is responsible for allocating a contiguous block of memory of a specified size, while the \textit{free} @@ -686,17 +686,17 @@ function deallocates the memory, returning it to the pool for future use. \end{algorithm} When the \textit{malloc} function (Algorithm \ref{alg:malloc}) is invoked, the algorithm employs an eager allocation strategy for physical memory. -This is achieved through the use of the SetBounds mechanism, which constructs a FAT specialized +This is achieved through the use of the SetBounds mechanism, which constructs a FAT-specialised pointer that encodes both the start and end addresses of the allocated memory region within the pointer itself. The start and end addresses correspond to the size of the memory block requested by \textit{malloc}. This -approach introduces a method of memory tracking, where the bounds of the allocated region are +approach introduces a method of memory tracking, where the bounds of the allocated region is explicitly encoded in the address, enabling efficient monitoring and management of memory usage. Furthermore, this design uses shared huge page TLB entries to map and track memory addresses. By encoding bounds directly into the address, the algorithm ensures that memory accesses remain within the allocated region, thereby reducing the risk of out-of-bounds -errors. This use of FAT and shared TLB entries not only aligns with the principles of -efficient memory management but also demonstrates a practical usecase of huge pages in CHERI. +errors. This use of FAT and shared TLB entries not only align with the principles of +efficient memory management but also demonstrate a practical use case of huge pages in CHERI. \begin{algorithm} \caption{Free implementation} @@ -925,37 +925,37 @@ patterns benefit from its design. The results align with expectations, showcasin of its capability to handle memory more efficiently by leveraging huge pages. \begin{itemize} - \item L1 DTLB reads: There was noticeable reduction of L1 DTLB reads for kmeans of about an average of - 4\% lesser and for Glibc there was significant reduction of 50\% lesser than Jemalloc. + \item L1 DTLB reads: There was a noticeable reduction of L1 DTLB reads for kmeans of about an average of + 4\% lesser and for Glibc there was significant reduction of 50\% less than Jemalloc. \item L2 DTLB reads: For all the benchmarks on figure \ref{fig:bargraph} there was on average 98\% - reduction on L2 DTLB reads. This demonstrates that all the TLB translations are read at the L1 TLB + reduction in L2 DTLB reads. This demonstrates that all the TLB translations are read at the L1 TLB cache. - \item DTLB walks: Due to most of the TLB entries getting hit at the L1 DTLB there is no need + \item DTLB walks: Due to most of the TLB entries getting hit at the L1 DTLB, there is no need to walk the TLB cache hierarchy. This is shown by an average of 99\% reduction in DTLB walks. \item L1 DTLB refills: Since there are fewer DTLB walks and most reads are done at the L1 DTLB layer there is no need for numerous TLB refills to take place. Our benchmarks show on average - a 99\% reduction on DTLB refills. + a 99\% reduction in DTLB refills. \end{itemize} A particularly striking observation is the significant reduction in data TLB walks, -L2 data TLB reads, and TLB refills—consistently showing a 90\% decrease across all +L2 data TLB reads, and TLB refills-consistently show a 90\% decrease across all benchmarks compared to Jemalloc. This improvement is due to the modified allocator's use of a single huge page entry at the L1 TLB layer. By enabling most address translations to be resolved directly at the L1 TLB, the need to walk through the deeper TLB hierarchy is largely eliminated. This reduction in translation overhead is a key factor in the allocator's performance for certain types of workloads. -The micro benchmarks, which are crafted to emphasize memory read operations, highlight the +The microbenchmarks, which are crafted to emphasize memory read operations, highlight the allocator's strengths. These tests simulate frequent and intensive memory access patterns, -where the reduction in TLB misses directly translates into measurable performance gains. +where the reduction in TLB misses directly translate into measurable performance gains. On average, the FAT allocator achieves a 50\% reduction in wall clock runtimes for these workloads, underscoring its ability to optimize high-throughput memory operations. -On the other hand, macro benchmarks, which represent larger and more complex real-world applications, +On the other hand, macro benchmarks, which represent larger and more complex real-world applications , exhibit minimal differences in wall clock runtimes when using the FAT allocator. This outcome is expected, as macro benchmarks typically involve a broader range of operations beyond memory allocation, diluting the impact of the allocator's optimizations. Additionally, @@ -1010,7 +1010,7 @@ consistent performance across varying cluster sizes in the K-means benchmark, wi minor anomalies observed under specific conditions. These outliers provide valuable insights into the allocator's interaction with system-level caching and memory translation mechanisms. -While the allocator excels in scenarios emphasizing high memory throughput, its impact on +While the allocator excels in scenarios emphasizing high-memory throughput, its impact on macro benchmarks is less pronounced. This suggests that its benefits are most relevant for applications with frequent and intensive memory operations rather than those constrained by computation or I/O bottlenecks. @@ -1039,7 +1039,7 @@ computation or I/O bottlenecks. \section{Conclusion} %Title of the Conclusion This paper addresses the growing disparity between application workloads and the capacity of TLBs. To mitigate this gap, we proposed leveraging physically contiguous memory with CHERI bounds to reduce TLB walks. -We designed a memory allocator which uses huge pages with CHERI CC scheme to track allocations within the +We designed a memory allocator that uses huge pages with the CHERI CC scheme to track allocations within the allocated huge page. This approach reduces the number of TLB entries needed while using bounds to minimize fragmentation. % Additionally, diff --git a/docs/EuroSys/Paper/paper.txt b/docs/EuroSys/Paper/paper.txt new file mode 100644 index 0000000..63f75a8 --- /dev/null +++ b/docs/EuroSys/Paper/paper.txt @@ -0,0 +1,655 @@ +--- +abstract: | + The increasing gap between workload memory requirements and the + capacity of translation lookaside buffers (TLBs) means TLB misses are + more frequency, costing additional clock cycles, impacting runtime + performance. One solution is to use physically contiguous memory in + conjunction with huge pages. We propose an alternative approach, by + exploiting capability-based addressing in the CHERI architecture. This + paper presents a new memory allocator. It associates capabilities with + memory pointers. integrating block-based allocations within huge + pages. Our allocator reduces TLB misses by up to 90%, which leads to + reduced runtimes for memory-intensive applications. +author: +- Akilan Selvacoumar +bibliography: +- paperReferences.bib +title: Fat Address Translations +--- + +# Introduction + +In computing, achieving high performance is an ongoing challenge, +especially as applications handle increasingly complex workloads. Memory +management is a key factor in performance, where efficient use of +resources is essential. Translation Lookaside Buffers (TLBs) are crucial +in this context, speeding up memory access by caching recent memory +address translations. A TLB, a specialised cache in the memory +management unit (MMU), reduces the time required to convert virtual +addresses to physical ones. When a program accesses data in memory, the +MMU first checks the TLB for a matching entry, avoiding the slower +process of consulting page tables. However, as applications grow larger +and more complex, the fixed size of TLBs often cannot keep up, leading +to more TLB misses and performance slowdowns [@mittal_survey_2017]. To +tackle this issue, researchers have explored new solutions, including +the use of huge pages [@panwar_hawkeye_2019]. + +Huge pages, also known as large pages, allow for the allocation of +memory in significantly larger chunks compared to traditional small +pages. By reducing the number of TLB entries needed to access a given +amount of memory, huge pages offer a potential avenue for optimising +TLBs use by reducing the number of entries needed to map large memory +regions. This not only decreases the frequency of TLB misses but also +lowers the overhead associated with address translation. By minimising +these bottlenecks, huge pages can improve system performance in aspects +such as speeding up memory-intensive applications, reducing latency in +data access, and enhancing throughput for workloads that rely heavily on +large datasets. + +Simultaneously, advancements in hardware-level security, such as the +Capability Hardware Enhanced RISC Instructions (CHERI) + [@woodruff_cheri_2014] architecture, present additional opportunities +for performance enhancement. CHERI's capability-based addressing +approach not only strengthens system security by tightly controlling +memory access but also opens avenues for optimising memory management +operations. By integrating CHERI's compressed encoded +bounds [@woodruff_cheri_2019] with the use of huge pages, We have shown +it is possible to track and manage large, physically contiguous memory +blocks without requiring numerous TLB entries. This combination reduces +TLB pressure by minimising the number of entries required to map +extensive memory regions, thereby decreasing TLB misses and improving +address translation performance. Furthermore, it accelerates +memory-intensive tasks by reducing the overhead associated with managing +non-contiguous memory allocations. The contributions for the following +paper are as follows: + +- **FAT Addresses Translations**: Introduces FAT that include memory + bounds, allowing efficient tracking and management of physically + contiguous memory regions (Section +  [3](#sec:FatPointerTranslations){reference-type="ref" + reference="sec:FatPointerTranslations"}). + +- **CHERI's Capability-based Optimization**: Demonstrates how CHERI's + architecture can be used to optimize memory allocation by encoding + memory bounds directly within pointers, reducing TLB reliance (Section +  [3.2](#sec:128bitCompressedBounds){reference-type="ref" + reference="sec:128bitCompressedBounds"}). + +- **Memory Allocation Algorithms**: Provides an algorithms for + allocating, freeing physically contiguous memory and integrating huge + pages with CHERI's capability-based bounds for enhanced memory + management (Section  [4](#sec:MemoryAllocator){reference-type="ref" + reference="sec:MemoryAllocator"}). + +Through comprehensive evaluation, including micro and macro benchmarks, +we demonstrate the allocator's ability to reduce TLB misses by up to +90%, yielding significant improvements in wall clock runtimes for +memory-intensive applications. While its impact on larger, +computation-heavy workloads is less pronounced, the proposed allocator +shows strong potential for advancing memory management in scenarios +requiring high memory throughput by reducing the address translation +overhead. + +# Related work {#sec:org0e192da} + +## Huge Pages + +Increasing TLB reach[@TLBReach] can be achieved by using larger page +sizes, such as huge pages [@panwar_hawkeye_2019], which are common in +modern computer systems. The x86-64 architecture supports huge pages of +2 MB and 1 GB, backed by OS mechanisms like Transparent Huge Pages +(THP) [@THP] and HugeTLBFS in Linux. However, available page sizes in +x86-64 are limited, leading to internal fragmentation issues. + +For instance, allocating 1 MB with 4 KB base pages requires 256 PTEs +(Page Table Entries), but using a 2 MB huge page would waste half of the +memory space. Some architectures offer more page size choices, such as +Intel Itanium, which allows different areas of the address space to have +their own page sizes. Itanium uses a hash page table to organize huge +pages, but without significant changes to the conventional page table, +it only helps reduce page walk overheads. Huge page tunable base page +size permits the OS to adjust the base page size, but still faces +internal fragmentation problems, with huge page recommending a base page +size of no more than 16 KB. Shadow Superpage [@Shadow_superpages] +introduces a new translation level in the memory controller to merge +non-contiguous physical pages into a huge page in a shadow memory space, +extending TLB coverage. However, this approach requires all memory +traffic to be translated again in the memory controller, resulting in +additional latency for memory accesses. + +## Direct Segment + +Early processors often used segments to manage virtual memory, where a +segment [@DirectSegment] essentially mapped contiguous virtual memory to +contiguous physical memory. Unlike pages, which are relatively small, +segments can be much larger, offering the potential for more efficient +memory management in certain scenarios. This concept of segmentation has +seen a resurgence in some modern approaches that aim to enhance +translation coverage by designating specific areas in the virtual +address space. + +This method allows programmers to explicitly define a single segment for +applications requiring significant memory. It introduces two new +registers to the system, which indicate the start and end of this +segment. Virtual addresses within this segment are translated by +calculating the offset from the virtual start address and applying this +offset to the physical start address. This straightforward method +simplifies the translation process for large memory areas but requires +significant modifications to the source code of applications. + +## Redundant Memory Mapping (RMM) + +Redundant Memory Mappings (RMM) [@karakostas_redundant_2015] enhance +memory management by introducing an additional range table that +pre-allocates contiguous physical pages for large memory allocations, +creating ranges that are both virtually and physically contiguous. This +approach simplifies address translation within these ranges by adding an +offset, similar to Direct Segment, but RMM supports multiple ranges and +operates transparently to programmers, requiring no source code +modifications. The range table, separate from the conventional page +table, holds the mappings for these large allocations. To determine +which range an address belongs to, RMM compares the address against all +range boundaries, a process that is computationally expensive and +therefore performed only after an L1 TLB miss. To optimize this, RMM +uses a range TLB (RTLB) to quickly identify if an address falls within +any pre-allocated range, facilitating efficient translation and reducing +overhead. Range mapping works alongside the paging system by generating +TLB entries on TLB misses and still performing TLB lookups for each +virtual address translation. Unlike traditional segmentation mechanisms, +range mapping activates a range lookaside buffer (RTLB) located with the +last level TLB upon a miss. The hardware TLB miss handler then searches +the RTLB for the miss address and, if found, generates a new TLB entry +with the physical address derived from the base virtual address and +range offset, along with permission bits. If the RTLB also misses, the +system defaults to a standard page walk while a range table walker +simultaneously loads the range into the RTLB in the background, avoiding +delays in memory operations. The RTLB, functioning as a fully +associative search structure, ensures that most last level TLB misses +are handled efficiently by range mapping, reducing the need for costly +page table walks. + +## CHERI {#sec:orgbf2eaac} + +CHERI extends conventional processor Instruction-Set Architectures +(ISAs) with architectural capabilities to enable fine-grained memory +protection and highly scalable software compartmentalization. It is a +hybrid capability architecture that can combine capabilities with +conventional MMU (Memory Management Unit) based systems. The +contributions of CHERI include ISA changes to introduce architectural +capabilities; a new microarchitecture that demonstrates capabilities can +be implemented efficiently in hardware, with support for efficient +tagged memory to protect capabilities and compress them to reduce memory +overhead; a newly designed software construction model that uses +capabilities to provide fine-grained memory protection and scalable +software compartmentalization; language and compiler extensions for +using capabilities with C and C++; and OS extensions to support +fine-grained memory protection (including spatial, referential, and +non-stack temporal memory safety) and abstraction extensions for +scalable software compartmentalization. + +## CHERI CC + +CHERI Concentrate: Practical Compressed +Capabilities[@woodruff_cheri_2019] introduces a compression scheme for +CHERI, aiming to address the performance and compatibility challenges +associated with capability pointers. Capability pointers enhance memory +safety by embedding bounds and permissions directly within pointers, but +traditional implementations double their size---leading to increased +memory usage. CHERI CC proposes a compression strategy that preserves +security while reducing size and inefficiencies. Key contributions +include a floating-point bounds encoding technique with an internal +exponent mechanism that offers greater precision for smaller objects and +optimized space usage for larger ones. + +# Fat Address Translations {#sec:FatPointerTranslations} + +Fat Address Translations (FAT) uses the CHERI architecture to bring +about block based allocations in physically contiguous memory. FAT +leverages techniques like FlexPointer [@chen_flexpointer_2023] and +RMM [@karakostas_redundant_2015] to reduce pressure on the TLB. A key +component in this implementation is the use of range addresses with +CHERI CC [@woodruff_cheri_2019]. + +![High overview +architecture](diagram/HighOverviewArchitecture.drawio.png){#fig:HighOverviewArchitecture +width="60%"} + +Figure [1](#fig:HighOverviewArchitecture){reference-type="ref" +reference="fig:HighOverviewArchitecture"} illustrates a comparison +between standard memory allocation (*malloc()*) and a proposed FAT +method. The standard approach involves a C program interacting with a +custom allocator which uses 48-bit virtual addresses and a TLB walker +(L1, L2 and L3 cache) to achieve non-contiguous allocation in physical +memory. This typically results in more TLB entries and increased TLB +misses increasing the reasoning to have more TLB walks. In contrast, the +FAT Address Translations method employs a custom allocator leveraging +physically contiguous memory by using CHERI to encode bounds within the +pointers and as shown in the figure +[1](#fig:HighOverviewArchitecture){reference-type="ref" +reference="fig:HighOverviewArchitecture"} there is almost no reliance on +walking the TLB hierarchy. + +## Encoding Ranges as Bounds to the Pointer {#sec:RangeMemory} + +![Range of memory](diagram/AllocationOverview24.png){#fig:RangeOfMemory +width="40%"} + +A memory range in FAT has two points to track memory in physical +contiguous space which is the top and bottom. These two points are two +virtual addresses and the range consists of addresses which lie within +this and refers to addresses allocated by invoking *malloc*. In FAT +memory ranges are established using bounds encoded within the pointer, +adhering to CHERI CC [@woodruff_cheri_2019]. + +Figure [2](#fig:RangeOfMemory){reference-type="ref" +reference="fig:RangeOfMemory"} illustrates a straightforward use-case in +which the dark pink line represents a single, large contiguous memory +area, or huge page. Within this huge page, the orange and blue lines +indicate two separate memory allocations equivalent to invoking *malloc* +twice to allocate memory in distinct regions. This scenario simulates a +block-based memory allocator operating within the confines of the huge +page. The allocations use the bounds encoded in the FAT, ensuring +tracking of the allocated memory regions. By using the CHERI bounds, +this method maintains the contiguity of the allocated blocks within the +huge page. + +## 128 bit compressed bounds {#sec:128bitCompressedBounds} + +We use CHERI CC [@woodruff_cheri_2019] to track regions of memory in +physically contiguous space. CHERI CC consists of compressed bounds that +represent a 128-bit pointer within a 64-bit virtual address system. Our +approach uses the work of CHERI CC by using a single cycle to decode +bounds from the capability register and the bounds which are decoded are +repurposed for tracking memory which is eagerly allocated. + +Allocators like Jemalloc typically allocate objects under 512 bytes. +When an object's bounds cannot be precisely represented, padding is +required to ensure memory safety. However, it has been observed that +Jemalloc rarely needs more than 6 bits to store the exponent values +within compressed bounds (as shown in [@woodruff_cheri_2019]). This +means that the default behavior of allocators such as Jemalloc, would +allow precise representation of bounds within CHERI CC. + +Instead of relying on fixed-size TLB entries with set page sizes (such +as 4KB, 2MB, or 1GB), FAT uses CHERI CC to define dynamic bounds based +on the size requested at allocation time (e.g., during a *malloc* call). +This approach offers a more flexible alternative to the traditional +fixed-size TLB model. + +## Instrumenting Block-Based Allocators with Physically Contiguous Memory + +![FAT Address Translations using huge +pages](diagram/TLBAccess.drawio.png){#fig:HugePages width="40%"} + +We are able to pre-allocate memory using huge pages and are able to mark +smaller allocations using ranges by storing them as bounds within the +pointer. Each of these memory ranges can be called as a block. Since we +have numerous blocks inside a huge page we allow block based memory +patters within physically contiguous memory. + +Figure [3](#fig:HugePages){reference-type="ref" +reference="fig:HugePages"} illustrates a use-case of huge pages where +the green line represents a sample access to read within a contigous +space of physical memory. The dotted lines represents the bounds for +that particular pointer access. Using bounds stored on the pointer a +block based pattern can be replicated on physically contigous memory. + +# Memory allocator design {#sec:MemoryAllocator} + +This section presents a straightforward memory allocator designed and +implemented based on the principles outlined FAT (Section + [3](#sec:FatPointerTranslations){reference-type="ref" +reference="sec:FatPointerTranslations"}). The allocator consists of +three core functions: *InitAlloc*, *malloc*, and *free*. The *InitAlloc* +function initializes the memory pool, setting up the necessary data +structures and metadata required for efficient memory management. The +*malloc* function is responsible for allocating a contiguous block of +memory of a specified size, while the *free* function deallocates the +memory, returning it to the pool for future use. + +:::: algorithm +::: algorithmic +$sz \gets \text{ALIGN\_UP}(sz, \text{MAX\_ALIGNMENT})$ +$\text{MallocCounter} \gets \text{MallocCounter} - sz$ +$\text{ptrLink} \gets \&\text{ptr}[\text{MallocCounter}]$ +$\text{ptrLink} \gets \text{SET\_BOUNDS}(\text{ptrLink}, sz)$ +$\text{ptrLink}$ +::: +:::: + +When the *malloc* function (Algorithm +[\[alg:malloc\]](#alg:malloc){reference-type="ref" +reference="alg:malloc"}) is invoked, the algorithm employs an eager +allocation strategy for physical memory. This is achieved through the +use of the SetBounds mechanism, which constructs a FAT specialized +pointer that encodes both the start and end addresses of the allocated +memory region within the pointer itself. The start and end addresses +correspond to the size of the memory block requested by *malloc*. This +approach introduces a method of memory tracking, where the bounds of the +allocated region are explicitly encoded in the address, enabling +efficient monitoring and management of memory usage. + +Furthermore, this design uses shared huge page TLB entries to map and +track memory addresses. By encoding bounds directly into the address, +the algorithm ensures that memory accesses remain within the allocated +region, thereby reducing the risk of out-of-bounds errors. This use of +FAT and shared TLB entries not only aligns with the principles of +efficient memory management but also demonstrates a practical usecase of +huge pages in CHERI. + +:::: algorithm +::: algorithmic +$\text{len} \gets \text{GET\_LENGTH}(\text{ptr})$ +$\text{UNMAP}(\text{ptr}, \text{len})$ +::: +:::: + +The memory deallocation (Algorithm +[\[alg:free\]](#alg:free){reference-type="ref" reference="alg:free"}) +mechanism in the proposed allocator is facilitated by the FAT structure +introduced in the *malloc* algorithm. When the *free* function is +invoked, it uses the metadata embedded within the FAT to determine the +range and size of the allocated memory region. Specifically, FAT encodes +the start and end addresses of each allocation, providing the +information needed to identify the memory block to be deallocated. This +enables the allocator to accurately unmap the corresponding memory +region from the address space. + +By extracting the bounds and size directly from FAT, the *free* function +eliminates the need for additional metadata lookups or complex data +structures. + +:::: algorithm +::: algorithmic +$\text{sz} \gets 1\ \text{GB}$ +$\text{fd} \gets \text{CREATE\_LARGE\_PAGE\_MEMORY}(\text{sz})$ +$\text{ptr} \gets \text{MAP\_MEMORY}(\text{sz})$ +$\text{MallocCounter} \gets \text{sz}$ +::: +:::: + +Algorithm [\[alg:initAlloc\]](#alg:initAlloc){reference-type="ref" +reference="alg:initAlloc"} describes the initialization of physically +contiguous memory through the use of huge pages, a mechanism supported +by modern architectures to optimize memory management. The algorithm +begins by allocating a fixed block of 1 GB of physically contiguous +memory. This decision is driven by the architectural constraints of +contemporary systems, particularly ARM-based CPUs, where 1 GB represents +the largest supported page size. By leveraging huge pages, the algorithm +reduces the overhead associated with page table management and enhances +memory access efficiency, which is critical for performance-sensitive +applications and kernel-level operations. + +# Evaluation {#sec:Evaluation} + +We conducted tests of the FAT memory allocator against +Jemalloc [@jemalloc], Jemalloc is the default memory allocator for +CHERIBSD [@cheribsd], to assess the performance improvements enabled by +the FAT allocator. Specifically, we evaluated the reduction in TLB walks +and misses and its impact on wall clock runtime. + +To comprehensively analyze the proposed allocator, we categorized +benchmarks into two classes which are micro and macro benchmarks. Micro +benchmarks comprise smaller C programs designed to target specific +allocator patterns, enabling us to evaluate detailed aspects of the +allocator's behavior. Macro benchmarks, on the other hand, encompass +larger, real-world C programs, allowing us to assess the allocator's +performance in more practical, real-world scenarios. + +## Experiment setup {#sec:Experiment} + +The CHERI Morello [@Morello] board was used to evaluate the proposed +memory allocator. Morello implements the ARM A76 with enhanced +server-class memory, featuring a quad-core ARM CPU with capability +extensions. The L1 and L2 caches were modified to proliferate the +capability bit, ensuring compatibility with CHERI's capability-based +memory model. When compiling the C programs for benchmarking, the +Benchmark ABI was used as recommended by the CHERI community. This +compilation mode was enabled using the Clang compiler. + +The Benchmark ABI [@BenchmarkABI] was specifically designed because the +Morello branch predictor was not expanded to predict bounds. +Consequently, a capability-based jump introduces stalls in later +PCC-dependent instructions until bounds are established. This issue is +particularly significant during dynamically linked calls and returns +between libraries, where bounds are changed to cover the called or +returned-to library. Such stalls can negatively affect performance, +making the Benchmark ABI an essential consideration for this evaluation. + +Each C program was executed using two different memory allocators. The +first was the modified C allocator, imported as a header file. This +approach was necessary because the Benchmark ABI shared object file +exhibited unexpected behavior, failing to overwrite the C program at +runtime with the intended *malloc* functions. The second allocator was +the standard OS memory allocator, which, in the case of CHERIBSD, is +Jemalloc. + +Performance measurements were carried out using ARM performance +counters [@PerformanceCounter] to ensure accurate evaluation. These +counters provided detailed metrics, allowing us to compare the +performance of the two allocators and assess the impact of the proposed +changes. + +::: table* + Performance counter Description + -------------------------------------------------- ------------------------------------------- + Wall clock The actual time taken from the start of a + computer program to the end. + + (p/l1d_tlb_rd) L1 data TLB reads Level 1 data TLB access, read + + (p/l2d_tlb_rd) L2 data TLB reads Level 2 data TLB access, read + + (p/l1d_tlb_refill) L1 data TLB refills Level 1 data TLB refill. + The Level 1 data TLB refill + counter tracks each access to + the L1D_TLB that results + in a refill of the Level 1 data + or unified TLB. This includes any + access that requires a memory lookup + due to a translation table walk + or accessing another level of TLB cache. + + (p/cpu_cycles) CPU cycles The CPU CYCLES counter increases with + every clock cycle. However, it can be + affected by changes in clock frequency, + such as when WFI (Wait for Interrupt) + or WFE (Wait for Event) + instructions pause the clock. + + (p/dtlb_walk) Data TLB walks Data TLB access with at least + one translation table walk. + + (p/ll_cache_miss_rd) Last level cache miss reads Last level cache miss, read + (This refers to every miss in the + Last level cache that occurs + during a memory read operation.) +::: + +## Benchmarks + +The benchmarks [@Benchmark] are classified into 2 classes: + +### Micro benchmark + +We further elaborated on the two classes of benchmarks executed. Micro +benchmarks (Section [\[sec:Micro\]](#sec:Micro){reference-type="ref" +reference="sec:Micro"}). focused on particular allocation and +deallocation patterns, such as sequential and random memory accesses, to +stress-test the allocator under controlled conditions. Macro benchmarks +involved real-world applications, offering insights into how the +allocator performs with complex memory allocation demands, large +datasets, and varying execution contexts. []{#sec:Micro +label="sec:Micro"} + +- `GLIBC`: The Glibc benchmark evaluates the performance of *malloc* and + *free* functions in single-threaded, multi-threaded, and emulated + multi-threading scenarios using various block sizes and allocation + patterns. It simulates real-world memory usage by partially + deallocating blocks in FIFO order and fully deallocating them in LIFO + order. Results are gathered across configurations to analyze + performance variations. + +- `MemAccess`: This benchmark by Alex Bordei evaluates the performance + impact of memory access patterns by constructing and traversing a + doubly linked list with varying working set sizes. It supports + sequential or randomized structures, optional node operations, and + multithreaded traversal using pthreads. The program dynamically + allocates memory and systematically doubles the working set size to + analyze memory hierarchy behavior. + +### Macro benchmark {#sec:Macro} + +- `Kmeans`: Kmeans implements a parallelized K-means clustering + algorithm that assigns data points to clusters based on proximity to + centroids, iteratively updating them until convergence. The + computation is distributed across threads using the pthread library, + dynamically assigning tasks to optimize performance. Parameters like + data size and clusters are configurable, and the program ensures + efficient memory management and synchronization. + +- `Richards`: Richards is a task scheduling benchmark that simulates a + multitasking environment with tasks of varying types and priorities, + communicating through queued packets. The schedule function manages + task execution based on state and priority, tracking processed packets + and held tasks for performance evaluation. Configurable iterations and + timing help measure system performance and ensure correctness. + +- `BARNES`: Implements the Barnes-Hut algorithm to efficiently simulate + the interactions within an $N$-body system. A comprehensive overview + of the Barnes-Hut method is provided by Singh in his doctoral + dissertation  [@singh1993]. The implementation we benchmark extends + the original method by permitting multiple particles to be stored + within each leaf cell of the spatial decomposition, enhancing + performance and scalability. This extension is described by Holt and + Singh  [@holt1995]. + +## Results {#sec:Results} + +![[]{#fig:bargraph label="fig:bargraph"}Percentage difference between +the modified memory allocator against the default system memory +allocator](diagram/bargraph.png){#fig:bargraph width=".9\\linewidth"} + +The graph (Figure [4](#fig:bargraph){reference-type="ref" +reference="fig:bargraph"}) highlights the performance comparison between +the modified memory allocator and Jemalloc, the default memory +allocator. The FAT memory allocator, specifically optimized for use with +huge pages, demonstrates a clear advantage in scenarios where memory +allocation patterns benefit from its design. The results align with +expectations, showcasing the impact of its capability to handle memory +more efficiently by leveraging huge pages. + +- L1 DTLB reads: There was noticeable reduction of L1 DTLB reads for + kmeans of about an average of 4% lesser and for Glibc there was + significant reduction of 50% lesser than Jemalloc. + +- L2 DTLB reads: For all the benchmarks on figure + [4](#fig:bargraph){reference-type="ref" reference="fig:bargraph"} + there was on average 98% reduction on L2 DTLB reads. This demonstrates + that all the TLB translations are read at the L1 TLB cache. + +- DTLB walks: Due to most of the TLB entries getting hit at the L1 DTLB + there is no need to walk the TLB cache hierarchy. This is shown by an + average of 99% reduction in DTLB walks. + +- L1 DTLB refills: Since there are fewer DTLB walks and most reads are + done at the L1 DTLB layer there is no need for numerous TLB refills to + take place. Our benchmarks show on average a 99% reduction on DTLB + refills. + +A particularly striking observation is the significant reduction in data +TLB walks, L2 data TLB reads, and TLB refills---consistently showing a +90% decrease across all benchmarks compared to Jemalloc. This +improvement is due to the modified allocator's use of a single huge page +entry at the L1 TLB layer. By enabling most address translations to be +resolved directly at the L1 TLB, the need to walk through the deeper TLB +hierarchy is largely eliminated. This reduction in translation overhead +is a key factor in the allocator's performance for certain types of +workloads. + +The micro benchmarks, which are crafted to emphasize memory read +operations, highlight the allocator's strengths. These tests simulate +frequent and intensive memory access patterns, where the reduction in +TLB misses directly translates into measurable performance gains. On +average, the FAT allocator achieves a 50% reduction in wall clock +runtimes for these workloads, underscoring its ability to optimize +high-throughput memory operations. + +On the other hand, macro benchmarks, which represent larger and more +complex real-world applications, exhibit minimal differences in wall +clock runtimes when using the FAT allocator. This outcome is expected, +as macro benchmarks typically involve a broader range of operations +beyond memory allocation, diluting the impact of the allocator's +optimizations. Additionally, the benefits of huge pages may be less +pronounced for these workloads, as they are often bottlenecked by +factors such as computation or I/O rather than memory translation +overhead. + +![[]{#fig:org8683315 label="fig:org8683315"}Kmeans COZ benchmark +executed against various cluster +sizes](./diagram/kmeans.png){#fig:org8683315 width="110%"} + +The K-means algorithm was executed with varying cluster sizes to +evaluate the performance difference between the FAT allocator and +Jemalloc as the workload scales. This analysis aims to understand how +the allocator's optimizations, particularly its ability to manage memory +more efficiently with huge pages, impact performance under different +workload conditions. + +For most cluster sizes tested, the percentage difference in performance +remained relatively consistent. This indicates that the allocator's +efficiency scales predictably with increasing workload sizes, suggesting +a stable and uniform benefit across different configurations. The +consistent performance gain is likely due to the allocator's ability to +minimize TLB misses and efficiently manage memory allocations for the +centroid and data point structures used in the K-means algorithm. + +However, an anomaly was observed at a cluster size of 2000, where the +percentage difference deviated significantly from the trend. At this +cluster size, the memory access patterns and allocation behavior may +align in a way that temporarily offsets the advantages of the FAT +allocator. For example, the memory layout might interact with +system-level caching mechanisms or TLB behavior differently, leading to +an unexpected change in performance. Additionally, the increased +complexity of managing a higher number of clusters might introduce +computational overhead that overshadows the memory allocator's +optimizations. + +## Analysis {#sec:Analysis} + +The FAT memory allocator demonstrates significant potential for +enhancing memory management in systems that benefit from huge page +optimizations. Its design effectively reduces TLB misses, achieving up +to 90% fewer data TLB walks, L2 TLB reads, and TLB refills compared to +Jemalloc. These improvements lead to noticeable performance gains, +especially in micro benchmarks, where the allocator reduces wall clock +runtimes by an average of 50%. + +The allocator integrates seamlessly into memory-intensive workloads, as +evidenced by its consistent performance across varying cluster sizes in +the K-means benchmark, with only minor anomalies observed under specific +conditions. These outliers provide valuable insights into the +allocator's interaction with system-level caching and memory translation +mechanisms. + +While the allocator excels in scenarios emphasizing high memory +throughput, its impact on macro benchmarks is less pronounced. This +suggests that its benefits are most relevant for applications with +frequent and intensive memory operations rather than those constrained +by computation or I/O bottlenecks. + +# Conclusion + +This paper addresses the growing disparity between application workloads +and the capacity of TLBs. To mitigate this gap, we proposed leveraging +physically contiguous memory with CHERI bounds to reduce TLB walks. We +designed a memory allocator which uses huge pages with CHERI CC scheme +to track allocations within the allocated huge page. This approach +reduces the number of TLB entries needed while using bounds to minimize +fragmentation.\ +The benchmarks demonstrate that the allocator reduces TLB misses by up +to 90%, leading to substantial performance gains in memory-intensive +workloads, though the improvements are less pronounced for larger, +computation-heavy applications. These results highlight the allocator's +potential to advance memory management by repurposing CHERI's +capability-based model with the use of huge pages. diff --git a/docs/EuroSys/Paper/todo.org b/docs/EuroSys/Paper/todo.org index 9d58be9..203a324 100644 --- a/docs/EuroSys/Paper/todo.org +++ b/docs/EuroSys/Paper/todo.org @@ -6,8 +6,6 @@ - [x] Space missing from bracket - [x] Utlizing to using - [x] Small number use word -- [ ] Check for unusual amount of commas and - String them into a sentence - [ ] Make all past tense to present tense - [x] Comment out future work section - [x] Section with always captial S