327
This mechanism was described in the previous section. Finally, maintaining fail-safety of
complex persistent data structures is expensive, and keeping them in DRAM allows the
allocator to sidestep that cost.
The runtime allocation scheme employed by libpmemobj is segregated fit with chunk
reuse and thread caching as described earlier. Free lists in libpmemobj, called buckets,
are placed in DRAM and are implemented as vectors of pointers to persistent memory
blocks. The persistent representation of this data structure is a bitmap, located at the
beginning of a larger buffer from which the smaller blocks are carved out. These buffers
in libpmemobj, called runs, are variably sized and are allocated from the previously
mentioned chunks. Very large allocations are directly allocated as chunks. Figure 16-7
shows the libpmemobj implementation.
Persistent allocators must also ensure consistency in the presence of failures,
otherwise, memory might become unreachable after an ungraceful shutdown of the
application. One part of the solution is the API we outlined in the previous section. The
other part is the careful design of the algorithms inside the allocator that ensures no
matter when the application is aborted, the state is consistent. This is also aided by redo
logs, which are used to ensure atomicity of groups of noncontiguous persistent metadata
changes.
Figure 16-7. On-media layout of libpmemobj’s heap
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures
This mechanism was described in the previous section. Finally, maintaining fail-safety of
complex persistent data structures is expensive, and keeping them in DRAM allows the
allocator to sidestep that cost.
The runtime allocation scheme employed by libpmemobj is segregated fit with chunk
reuse and thread caching as described earlier. Free lists in libpmemobj, called buckets,
are placed in DRAM and are implemented as vectors of pointers to persistent memory
blocks. The persistent representation of this data structure is a bitmap, located at the
beginning of a larger buffer from which the smaller blocks are carved out. These buffers
in libpmemobj, called runs, are variably sized and are allocated from the previously
mentioned chunks. Very large allocations are directly allocated as chunks. Figure 16-7
shows the libpmemobj implementation.
Persistent allocators must also ensure consistency in the presence of failures,
otherwise, memory might become unreachable after an ungraceful shutdown of the
application. One part of the solution is the API we outlined in the previous section. The
other part is the careful design of the algorithms inside the allocator that ensures no
matter when the application is aborted, the state is consistent. This is also aided by redo
logs, which are used to ensure atomicity of groups of noncontiguous persistent metadata
changes.
Figure 16-7. On-media layout of libpmemobj’s heap
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures