177
Child processes created using the fork(2) system call inherit the MAP_PRIVATE
mappings from the parent process. When memory pages are modified by the parent
process, a copy-on-write mechanism is triggered by the kernel to create an unmodified
copy for the child process. These pages are allocated on the same NUMA node as the
original page.
libvmemcache: An Efficient Volatile Key-Value
Cache for Large-Capacity Persistent Memory
Some existing in-memory databases (IMDB) rely on manual dynamic memory allocations
(malloc, jemalloc, tcmalloc), which can exhibit external and internal memory
fragmentation when run for a long period of time, leaving large amounts of memory
un- allocatable. Internal and external fragmentation is briefly explained as follows:
• Internal fragmentation occurs when more memory is allocated
than is required, and the unused memory is contained within the
allocated region. For example, if the requested allocation size is 200
bytes, a chunk of 256 bytes is allocated.
• External fragmentation occurs when variable memory sizes are
allocated dynamically, resulting in a failure to allocate a contiguous
chunk of memory, although the requested chunk of memory remains
available in the system. This problem is more pronounced when large
capacities of persistent memory are being used as volatile memory.
Applications with substantially long runtimes need to solve this
problem, especially if the allocated sizes have considerable variation.
Applications and runtime environments handle this problem in
different ways, for example:
• Java and .NET use compacting garbage collection
• Redis and Apache Ignite* use defragmentation algorithms
• Memcached uses a slab allocator
Each of the above allocator mechanisms has pros and cons. Garbage collection and
defragmentation algorithms require processing to occur on the heap to free unused
allocations or move data to create contiguous space. Slab allocators usually define a fixed
set of different sized buckets at initialization without knowing how many of each bucket
Chapter 10 Volatile Use of persistent MeMory
Child processes created using the fork(2) system call inherit the MAP_PRIVATE
mappings from the parent process. When memory pages are modified by the parent
process, a copy-on-write mechanism is triggered by the kernel to create an unmodified
copy for the child process. These pages are allocated on the same NUMA node as the
original page.
libvmemcache: An Efficient Volatile Key-Value
Cache for Large-Capacity Persistent Memory
Some existing in-memory databases (IMDB) rely on manual dynamic memory allocations
(malloc, jemalloc, tcmalloc), which can exhibit external and internal memory
fragmentation when run for a long period of time, leaving large amounts of memory
un- allocatable. Internal and external fragmentation is briefly explained as follows:
• Internal fragmentation occurs when more memory is allocated
than is required, and the unused memory is contained within the
allocated region. For example, if the requested allocation size is 200
bytes, a chunk of 256 bytes is allocated.
• External fragmentation occurs when variable memory sizes are
allocated dynamically, resulting in a failure to allocate a contiguous
chunk of memory, although the requested chunk of memory remains
available in the system. This problem is more pronounced when large
capacities of persistent memory are being used as volatile memory.
Applications with substantially long runtimes need to solve this
problem, especially if the allocated sizes have considerable variation.
Applications and runtime environments handle this problem in
different ways, for example:
• Java and .NET use compacting garbage collection
• Redis and Apache Ignite* use defragmentation algorithms
• Memcached uses a slab allocator
Each of the above allocator mechanisms has pros and cons. Garbage collection and
defragmentation algorithms require processing to occur on the heap to free unused
allocations or move data to create contiguous space. Slab allocators usually define a fixed
set of different sized buckets at initialization without knowing how many of each bucket
Chapter 10 Volatile Use of persistent MeMory