349
The RDMA protocols are governed by the RDMA Wire Protocol Standards, which are
driven by the IBTA (InfiniBand Trade Association) and the IEFT (Internet Engineering
Task Force) specifications. The IBTA (https://www.infinibandta.org/) governs the
InfiniBand and RoCE protocols, while the IEFT (https://www.ietf.org/) governs
iWARP.
Low-latency RDMA networking protocols allow the network interface controller
(NIC) to control the movement of data between an initiator node source buffer and the
sink buffer on the target node without needing either node’s CPU to be involved in the
data movement. In fact, RDMA Read and RDMA Write operations are often referred
to as one-sided operations because all of the information required to move the data is
supplied by the initiator and the CPU on the target node is not typically interrupted or
even aware of the data transfer.
To perform remote data transfers, information from the target node’s buffers must
be passed to the initiator before the remote operation( s) will begin. This requires
configuring the local initiator’s RDMA resources and buffers. Similarly, the remote target
node’s RDMA resources that will require CPU resources will need to be initialized and
reported to the initiator. However, once the resources for the RDMA transfers are set up
and applications initiate the RDMA request using the CPU, the NIC does the actual data
movement on behalf of the RDMA-aware application.
RDMA-aware applications are responsible for:
• Interrogating each NIC on every initiator and target system to
determine supported features
• Selecting a NIC for each end of the RDMA point-to-point connection
• Creating the connection with the selected NICs, described as an
RDMA protection domain
• Allocating queues for the incoming and outgoing message on each
NIC and assigning those hardware resources to the protection domain
• Allocating DRAM or persistent memory buffers for use with RDMA,
registering those buffers with the NIC, and assigning those buffers to
the protection domain
Chapter 18 remote persistent memory
The RDMA protocols are governed by the RDMA Wire Protocol Standards, which are
driven by the IBTA (InfiniBand Trade Association) and the IEFT (Internet Engineering
Task Force) specifications. The IBTA (https://www.infinibandta.org/) governs the
InfiniBand and RoCE protocols, while the IEFT (https://www.ietf.org/) governs
iWARP.
Low-latency RDMA networking protocols allow the network interface controller
(NIC) to control the movement of data between an initiator node source buffer and the
sink buffer on the target node without needing either node’s CPU to be involved in the
data movement. In fact, RDMA Read and RDMA Write operations are often referred
to as one-sided operations because all of the information required to move the data is
supplied by the initiator and the CPU on the target node is not typically interrupted or
even aware of the data transfer.
To perform remote data transfers, information from the target node’s buffers must
be passed to the initiator before the remote operation( s) will begin. This requires
configuring the local initiator’s RDMA resources and buffers. Similarly, the remote target
node’s RDMA resources that will require CPU resources will need to be initialized and
reported to the initiator. However, once the resources for the RDMA transfers are set up
and applications initiate the RDMA request using the CPU, the NIC does the actual data
movement on behalf of the RDMA-aware application.
RDMA-aware applications are responsible for:
• Interrogating each NIC on every initiator and target system to
determine supported features
• Selecting a NIC for each end of the RDMA point-to-point connection
• Creating the connection with the selected NICs, described as an
RDMA protection domain
• Allocating queues for the incoming and outgoing message on each
NIC and assigning those hardware resources to the protection domain
• Allocating DRAM or persistent memory buffers for use with RDMA,
registering those buffers with the NIC, and assigning those buffers to
the protection domain
Chapter 18 remote persistent memory