322
libpmemobj Unified Logging
Both redo and undo logging in libpmemobj share the same internal interface and data
structure, which is called a unified log (or ulog for short). This is because redo and undo
logging only differ in the execution order of the log phases, or more precisely, when
the log is applied on commit or recovery. In practice, however, there are performance
considerations that require specialization in certain parts of the algorithm.
The ulog data structure contains one cache line header with metadata and a variable
length array of data bytes. The header consists of:
• A checksum for both the header and data, used only for redo logs
• A monotonically increasing generation number of a transaction in
the log, used only for undo logs
• The total length in bytes of the data array
• An offset of the next log in the group
The last field is used to create a singly linked list of all logs that participate in a single
transaction. This is because it is impossible to predict the total required size of the log
at the beginning of the transaction, so the library cannot allocate a log structure that is
the exact required length ahead of time. Instead, the logs are allocated on demand and
atomically linked into a list.
The unified log supports two ways of fail-safe inserting of entries:
1. Bulk insert takes an array of log entries, prepares the header of
the log, and creates a checksum of both the header and data. Once
done, a non-temporal copy, followed by a fence, is performed
to store this structure into persistent memory. This is the way in
which a group of deferred memory modifications forms a redo
log with only one additional barrier at the end of the transaction.
In this case, the checksum in the header is used to verify the
consistency of the entire log. If that checksum doesn’t match, the
log is skipped during recovery.
2. Buffer insert takes only a single entry, checksums it together
with the current generation number, and stores it in persistent
memory through non-temporal stores followed by a fence. This
method is used to create undo logs when snapshotting. Undo logs
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures
libpmemobj Unified Logging
Both redo and undo logging in libpmemobj share the same internal interface and data
structure, which is called a unified log (or ulog for short). This is because redo and undo
logging only differ in the execution order of the log phases, or more precisely, when
the log is applied on commit or recovery. In practice, however, there are performance
considerations that require specialization in certain parts of the algorithm.
The ulog data structure contains one cache line header with metadata and a variable
length array of data bytes. The header consists of:
• A checksum for both the header and data, used only for redo logs
• A monotonically increasing generation number of a transaction in
the log, used only for undo logs
• The total length in bytes of the data array
• An offset of the next log in the group
The last field is used to create a singly linked list of all logs that participate in a single
transaction. This is because it is impossible to predict the total required size of the log
at the beginning of the transaction, so the library cannot allocate a log structure that is
the exact required length ahead of time. Instead, the logs are allocated on demand and
atomically linked into a list.
The unified log supports two ways of fail-safe inserting of entries:
1. Bulk insert takes an array of log entries, prepares the header of
the log, and creates a checksum of both the header and data. Once
done, a non-temporal copy, followed by a fence, is performed
to store this structure into persistent memory. This is the way in
which a group of deferred memory modifications forms a redo
log with only one additional barrier at the end of the transaction.
In this case, the checksum in the header is used to verify the
consistency of the entire log. If that checksum doesn’t match, the
log is skipped during recovery.
2. Buffer insert takes only a single entry, checksums it together
with the current generation number, and stores it in persistent
memory through non-temporal stores followed by a fence. This
method is used to create undo logs when snapshotting. Undo logs
Chapter 16 pMDK Internals: IMportant algorIthMs anD Data struCtures