358
On Linux, the Open Fabric Alliance (OFA) libibverbs library provides ring-3 interfaces
to configure and use the RDMA connection for NICs that support IB, RoCE, and iWARP
RDMA network protocols. The OFA libfabric ring-3 application library can be layered
on top of libibverbs to provide a generic high-level common API that can be used with
typical RDMA NICs. This common API requires a provider plug-in to implement the
common API for the specific network protocol. The OFA web site contains many example
applications and performance tests that can be used on Linux with a variety of RDMAcapable NICs. Those examples provide the backbone of the PMDK librpmem library.
Windows implements remotely mounted NTFS volumes via the ring-3 SMB Direct
Application library, which provides a number of storage protocols including block
storage over RDMA.
Figure 18-4 provides the basic high-level architecture for a typical RDMA application
on Linux, using all of the publically available libraries and interfaces. Notice that a separate
side-band connection is typically needed to set up the RDMA connections themselves.
librpmem Architecture and Its Use in Replication
PMDK implements both the general-purpose remote replication method and the
appliance remote replication method in the librpmem library. As of PMDK v1.7, the
librpmem library implements the synchronous and asynchronous replication of local
writes to persistent memory on remote systems. librpmem is a low-level library, like
libpmem, which allows other libraries to use its replication features.
Figure 18-4. General RDMA software architecture
Chapter 18 remote persistent memory
On Linux, the Open Fabric Alliance (OFA) libibverbs library provides ring-3 interfaces
to configure and use the RDMA connection for NICs that support IB, RoCE, and iWARP
RDMA network protocols. The OFA libfabric ring-3 application library can be layered
on top of libibverbs to provide a generic high-level common API that can be used with
typical RDMA NICs. This common API requires a provider plug-in to implement the
common API for the specific network protocol. The OFA web site contains many example
applications and performance tests that can be used on Linux with a variety of RDMAcapable NICs. Those examples provide the backbone of the PMDK librpmem library.
Windows implements remotely mounted NTFS volumes via the ring-3 SMB Direct
Application library, which provides a number of storage protocols including block
storage over RDMA.
Figure 18-4 provides the basic high-level architecture for a typical RDMA application
on Linux, using all of the publically available libraries and interfaces. Notice that a separate
side-band connection is typically needed to set up the RDMA connections themselves.
librpmem Architecture and Its Use in Replication
PMDK implements both the general-purpose remote replication method and the
appliance remote replication method in the librpmem library. As of PMDK v1.7, the
librpmem library implements the synchronous and asynchronous replication of local
writes to persistent memory on remote systems. librpmem is a low-level library, like
libpmem, which allows other libraries to use its replication features.
Figure 18-4. General RDMA software architecture
Chapter 18 remote persistent memory