351
since the target application has no other context of what data movement has taken place.
For example, because there is no good way to know when writes have completed on
the target, an RDMA Send is often used to notify the target node what is happening. For
small amounts of data, the RDMA Send is very efficient, but it always requires targetside interaction to complete. An RDMA Write with immediate data operation will also
allow the target node to be interrupted when the write has completed as a different
mechanism for bookkeeping.
Goals of the Initial Remote Persistent Memory
Architecture
The goal of the first remote persistent memory implementation was based on minimal
changes – or ideally, no changes – to the current RDMA hardware and software stacks
used with volatile memory. From a network hardware, middleware, and software
architecture standpoint, writing to remote volatile memory is identical to writing to
remote persistent memory. The knowledge that a specific memory-mapped file is
backed by persistent memory vs. volatile memory is entirely the responsibility of the
application to maintain. None of the lower layers in the networking stack are aware of the
fact that the write is to a persistent memory region or volatile memory. The responsibility
of knowing which write persistence method to use for a given target connection, and
making those remote writes persistent, falls to the application.
Guaranteeing Remote Persistence
Until this chapter, much of the book focuses on the use and programming of persistent
memory on the local machine. You are now aware of some of the challenges of using
persistent memory, the persistence domain, and the need to understand and use a
flushing mechanism to ensure the data is persistent. These same programming concepts
and challenges apply to remote persistent memory with the additional constraints of
making it work within the existing network protocol and network latency.
The SNIA NVM programming model (described in Chapter 3) requires applications
to flush data that has been written to persistent memory to guarantee that the written
data made it into the persistence domain. This same requirement applies to writing to
remote persistent memory. After the RDMA Write or Send operation has moved the data
Chapter 18 remote persistent memory
since the target application has no other context of what data movement has taken place.
For example, because there is no good way to know when writes have completed on
the target, an RDMA Send is often used to notify the target node what is happening. For
small amounts of data, the RDMA Send is very efficient, but it always requires targetside interaction to complete. An RDMA Write with immediate data operation will also
allow the target node to be interrupted when the write has completed as a different
mechanism for bookkeeping.
Goals of the Initial Remote Persistent Memory
Architecture
The goal of the first remote persistent memory implementation was based on minimal
changes – or ideally, no changes – to the current RDMA hardware and software stacks
used with volatile memory. From a network hardware, middleware, and software
architecture standpoint, writing to remote volatile memory is identical to writing to
remote persistent memory. The knowledge that a specific memory-mapped file is
backed by persistent memory vs. volatile memory is entirely the responsibility of the
application to maintain. None of the lower layers in the networking stack are aware of the
fact that the write is to a persistent memory region or volatile memory. The responsibility
of knowing which write persistence method to use for a given target connection, and
making those remote writes persistent, falls to the application.
Guaranteeing Remote Persistence
Until this chapter, much of the book focuses on the use and programming of persistent
memory on the local machine. You are now aware of some of the challenges of using
persistent memory, the persistence domain, and the need to understand and use a
flushing mechanism to ensure the data is persistent. These same programming concepts
and challenges apply to remote persistent memory with the additional constraints of
making it work within the existing network protocol and network latency.
The SNIA NVM programming model (described in Chapter 3) requires applications
to flush data that has been written to persistent memory to guarantee that the written
data made it into the persistence domain. This same requirement applies to writing to
remote persistent memory. After the RDMA Write or Send operation has moved the data
Chapter 18 remote persistent memory