324
All memory operations, called “actions” in the API, are broken up into two individual
steps.
The first step reserves the state that is needed to perform the operation. For
allocations, this means retrieving a free memory block, marking it as reserved, and
initializing the object’s content. This reservation is stored in a user-provided runtime
variable. The library guarantees that if an application crashes while holding reservations,
the persistent state is not affected. That is why these action variables must not be
persistent.
The second step is the act of exercising the reservations, which is called
“publication.” Reservations can be published individually, but the true power of this API
lies in its ability to group and publish many different actions together.
The internal allocator API also has a function to create an action that will set a
memory location to a given value when published. This is used to modify the destination
pointer value and is instrumental in making the atomic API of libpmemobj fail-safe.
All internal allocator APIs that need to perform fail-safe atomic actions take
operation context as an argument, which is the runtime instance of a single log. It
contains various state information, such as the total capacity of the log and the current
number of entries. It exposes the functions to create either bulk or singular log entries.
The allocator’s functions will log and then process all metadata modifications inside of
the persistent log that belongs to the provided instance of the operating context.
Persistent Memory Heap Management: Allocator
Design for Persistent Memory
The previous section described the interface for the memory allocation used internally
in libpmemobj, but that was only the tip of the allocator iceberg. Before diving deeper
into this topic, we briefly describe the principles behind normal volatile allocators so you
can understand how persistent memory impacts the status quo.
Traditional allocators for volatile memory are responsible for efficient – in both time
and space – management of operating system–provided memory pages. Precisely how
this should be done for the generic case is an active research area of computer science;
many different techniques can be used. All of them try to exploit the regularities in
allocation and deallocation patterns to minimize heap fragmentation.
Most commonly used general-purpose memory allocators settled on an algorithm
that we refer to as “segregated fit with page reuse and thread caching.”
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures
All memory operations, called “actions” in the API, are broken up into two individual
steps.
The first step reserves the state that is needed to perform the operation. For
allocations, this means retrieving a free memory block, marking it as reserved, and
initializing the object’s content. This reservation is stored in a user-provided runtime
variable. The library guarantees that if an application crashes while holding reservations,
the persistent state is not affected. That is why these action variables must not be
persistent.
The second step is the act of exercising the reservations, which is called
“publication.” Reservations can be published individually, but the true power of this API
lies in its ability to group and publish many different actions together.
The internal allocator API also has a function to create an action that will set a
memory location to a given value when published. This is used to modify the destination
pointer value and is instrumental in making the atomic API of libpmemobj fail-safe.
All internal allocator APIs that need to perform fail-safe atomic actions take
operation context as an argument, which is the runtime instance of a single log. It
contains various state information, such as the total capacity of the log and the current
number of entries. It exposes the functions to create either bulk or singular log entries.
The allocator’s functions will log and then process all metadata modifications inside of
the persistent log that belongs to the provided instance of the operating context.
Persistent Memory Heap Management: Allocator
Design for Persistent Memory
The previous section described the interface for the memory allocation used internally
in libpmemobj, but that was only the tip of the allocator iceberg. Before diving deeper
into this topic, we briefly describe the principles behind normal volatile allocators so you
can understand how persistent memory impacts the status quo.
Traditional allocators for volatile memory are responsible for efficient – in both time
and space – management of operating system–provided memory pages. Precisely how
this should be done for the generic case is an active research area of computer science;
many different techniques can be used. All of them try to exploit the regularities in
allocation and deallocation patterns to minimize heap fragmentation.
Most commonly used general-purpose memory allocators settled on an algorithm
that we refer to as “segregated fit with page reuse and thread caching.”
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures