Intel Corporation

Intel-ne, Inc.

Compaq Computer Corporation, Inc.

Kenneth G Keels

Brian S Hausauer

Robert O Sharp

Vadim Makhervaks

John S Lacombe

Donald Wood

Jeff M Carlson

David J Maguire

Philip John Brisky

Shaun Wandler

Ali S Oztaskin

Jose Niell

Patrick Fleming

Steen K Larsen

Robert W Cone

Chih-Jen Chang

Ygdal Naouri

Brad Burres

Dan Biederman

Russell L Gillenwater

Eric W Multanen

Pat Wang

Matthew Akers

Tristan T Gross

Eric Rose

Eric Jon Schneider

Russel L Gillenwater

Technologies for an accelerator interface over Ethernet are disclosed. In the illustrative embodiment, a network interface controller of a compute device may receive a data packet. If the network interface controller determines that the data packet should be pre-processed (e.g., decrypted) with a remote accelerator device, the network interface controller may encapsulate the data packet in an encapsulating network packet and send the encapsulating network packet to a remote accelerator device on a remote compute device. The remote accelerator device may pre-process the data packet (e.g., decrypt the data packet) and send it back to the network interface controller. The network interface controller may then send the pre-processed packet to a processor of the compute device.

Technologies for accelerator interface

Technologies for remote direct memory access (RDMA) congestion control include a requester device and a responder device in communication over an Ethernet network. The requester device sends routable RDMA packets to the responder device over the Ethernet network. The packets may be RDMA over Converged Ethernet version 2 (RoCEv2) packets. The responder device determines whether any of the received packets have been marked by the network with a congestion encountered codepoint. If so, the responder device sends an acknowledgment packet with an express congestion notification bit set in the RDMA base transport header. The requester device updates a congestion window as a function of a number of congested packets acknowledged and a total number of packets acknowledged. Those operations may be performed by a network controller of each of the requester device and the responder device. Other embodiments are described and claimed.

Technologies for congestion control for IP-routable RDMA over converged ethernet

Methods, apparatus and software for implementing enhanced data center congestion management for non-TCP traffic. Non-congested transit latencies are determined for transmission of packets or Ethernet frames along paths between source and destination end-end-nodes when congestion along the paths is not present or minimal. Transit latencies are similarly measured along the same source-destination paths during ongoing operations during which traffic congestion may vary. Based on whether a difference between the transit latency for a packet or frame and the non-congested transit latency for the path exceeds a threshold, the path is marked as congested or not congested. A rate at which the non-TCP packets are transmitted along the path is then managed as function of a rate at which the path is marked as congested. In one implementation, non-TCP traffic is managed by mimicking a Data Center TCP technique, under which the congestion marking status of the path is substituted as an input to a DCTP algorithm in place of the normally-used ECN-Echo flag input. The congestion window output by the DCTCP algorithm is then used to manage the rate at which non-TCP packets to be forwarded via the path are transmitted from a source end-node.

Data center congestion management for non-TCP traffic

In an embodiment of the present invention, a method includes partitioning a plurality of remote direct memory access context objects among a plurality of virtual functions, establishing a remote direct memory access connection between a first of the plurality of virtual functions, and migrating the remote direct memory access connection from the first of the plurality of virtual functions to a second of the plurality of virtual functions without disconnecting from the remote peer.

Remote direct memory access adapter state migration in a virtual environment

Generally, this disclosure relates to a shared send queue in a networked system. A method, apparatus and system are configured to support a plurality of reliable communication channels using a shared send queue. The reliable communication channels are configured to carry messages from a host to a plurality of destinations and to ensure completed order of messages is related to a transmission order.

Shared send queue

Apparatus, method and system for supporting Remote Direct Memory Access (RDMA) Read V2 Request and Response messages using the Internet Wide Area RDMA Protocol (iWARP). iWARP logic in an RDMA Network Interface Controller (RNIC) is configured to generate a new RDMA Read V2 Request message and generate a new RDMA Read V2 Response message in response to a received RDMA Read V2 Request message, and send the messages to an RDMA remote peer using iWARP implemented over an Ethernet network. The iWARP logic is further configured to process RDMA Read V2 Response messages received from the RDMA remote peer, and to write data contained in the messages to appropriate locations using DMA transfers from buffers on the RNIC into system memory. In addition, the new semantics removes the need for extra operations to grant and revoke remote access rights.

IWARP RDMA read extensions

Methods, apparatus and systems for reducing usage of Doorbell Rings in connection with RDMA operations. A portion of system memory is employed as a Memory-Mapped Input/Output (MMIO) address space configured to be accessed via a hardware networking device. A Send Queue (SQ) is stored in MMIO and is used to facilitate processing of Work Requests (WRs) that are written to SQ entries by software and read from the SQ via the hardware networking device. The software and logic in the hardware networking device employ pointers identifying locations in the SQ corresponding to a next write WR entry slot and last read WR entry slot that are implemented to enable WRs to be written to and read from the SQ during ongoing operations under which the SQ is not emptied such that doorbell rings to notify the hardware networking device that new WRs have been written to the SQ are not required.

Methods, apparatus and systems for facilitating RDMA operations with reduced doorbell rings

Apparatus, methods and systems for supporting Send with Immediate Data messages using Remote Direct Memory Access (RDMA) and the Internet Wide Area RDMA Protocol (iWARP). iWARP logic in an RDMA Network Interface Controller (RNIC) is configured to generate different types of Send with Immediate Data messages, each including a header with a unique RDMA opcode identifying the type of Send with Immediate Data message, and send the message to an RDMA remote peer using iWARP implemented over an Ethernet network. The iWARP logic is further configured to process the Send with Immediate Data messages received from the RDMA remote peer. The Send with Immediate Data messages include a Send with Immediate Data message, a Send with Invalidate and Immediate Data message, a Send with Solicited Event (SE) and Immediate Data message, and a Send with Invalidate and SE and Immediate Data message.

iWARP send with immediate data operations

An embodiment may include circuitry that may write a message from a system memory in a host to a memory space in an input/output (I/O) controller in the host. A host operating system may reside, at least in part, in the system memory. The message may include both data and at least one descriptor associated with the data. The data may be included in the at least one descriptor. The circuitry also may signal the I/O controller that the writing has occurred.

Writing message to controller memory space

An apparatus is provided, for performing a direct memory access (DMA) operation between a host memory in a first server and a network adapter. The apparatus includes a host frame parser and a protocol engine. The host frame parser is configured to receive data corresponding to the DMA operation from a host interface, and is configured to insert markers on-the-fly into the data at a prescribed interval and to provide marked data for transmission to a second server over a network fabric. The protocol engine is coupled to the host frame parser. The protocol engine is configured to direct the host frame parser to insert the markers, and is configured to specify a first marker value and an offset value, whereby the host frame parser is enabled to locate and insert a first marker into the data.

Apparatus and method for in-line insertion and removal of markers

A flexible arrangement allows a single arrangement of Ethernet channel adapter (ECA) hardware functions to appear as needed to conform to various operating system deployment models. A PCI interface presents a logical model of virtual devices appropriate to the relevant operating system. Mapping parameters and values are associated with the packet streams to allow the packet streams to be properly processed according to the presented logical model and needed operations. Mapping occurs at both the host side and at the network side to allow the multiple operations of the ECA to be performed while still allowing proper delivery at each interface.

Method and apparatus for using a single multi-function adapter with different operating systems

A mechanism for performing remote direct memory access (RDMA) operations between a first server and a second server over an Ethernet fabric. The RDMA operations are initiated by execution of a verb according to a remote direct memory access protocol. The verb is executed by a CPU on the first server. The apparatus includes transaction logic that is configured to process a work queue element corresponding to the verb, and that is configured to accomplish the RDMA operations over a TCP/IP interface between the first and second servers, where the work queue element resides within first host memory corresponding to the first server. The transaction logic includes transmit history information stores and a protocol engine. The transmit history information stores maintains parameters associated with said work queue element. The protocol engine is coupled to the transmit history information stores and is configured to access the parameters to enable retransmission of one or more TCP segments corresponding to the RDMA operations.

Growing community of inventors

Georgetown, TX, United States of America

Kenneth G Keels