Intel Corporation

International Business Machines Corporation

Habanero Holdings, Inc.

Oracle International Corporation

Sequent Computer Systems, Inc.

At&t Bell Laboratories

Fabric7 Systems, Inc.

Thomas D Lovett

Nakul Pratap Saraiya

Cosmos Nicolaou

Shreyas Shah

Sharad Mehrotra

Bruce Michael Gilbert

Albert S Cheng

Michael A Parker

Todd M Rimmer

Donald R DeSota

Mangesh Shingane

Yuri Finkelstein

Wayne Alan Downer

Curtis M Collins

Radia Joy Perlman

Anil Vasudevan

Robert J Safranek

Ilango S Ganga

Hugh Wilkinson

Mark S Birrittella

Greg J Regnier

Alain Gravel

Myron H White

Shyam Mittur

Rajesh K Jagannathan

Geoffrey H Hanson

Fabio Onofrio Ingrao

Maged M Michael

Mehul M Shah

Thomas Benjamin Berg

Robert Zak

Robert Joersz

Steven F Hoover

Robert T Joersz

David Pardo Keppel

Donald E Blahut

Gary S Muntz

Alberto J Munoz

James Kunz

Gregory J Hubbard

Russell M Clapp

Daniel H Bax

Allen Hirashiki

William Jackson Bibb, Jr

Kenneth Frank Dove

Daryl V McDaniel

Doug Joseph

Michael S Parker

Peter V LaMaster

Brian W Colbry

Tom Gourley

James A Kunz

This disclosure describes systems, devices, methods and computer readable media for enhanced network communication for use in higher performance applications including storage, high performance computing (HPC) and Ethernet-based fabric interconnects. In some embodiments, a network controller may include a transmitter circuit configured to transmit packets on a plurality of virtual lanes (VLs), the VLs associated with a defined VL priority and an allocated share of network bandwidth. The network controller may also include a bandwidth monitor module configured to measure bandwidth consumed by the packets and an arbiter module configured to adjust the VL priority based on a comparison of the measured bandwidth to the allocated share of network bandwidth. The transmitter circuit may be further configured to transmit the packets based on the adjusted VL priority.

Traffic class arbitration based on priority and bandwidth allocation

Technologies for Ethernet gateway congestion management in HPC architectures include a high-performance computing (HPC) switch with an Ethernet gateway that is configured to receive an HPC packet from an HPC fabric via a virtual lane (VL) of the Ethernet gateway. The Ethernet gateway is further configured to determine whether the HPC packet corresponds to a backward error correction notification (BECN), identify one or more priority code points (PCPs) of the HPC packet corresponding to a BECN as a function of the VL on which the HPC packet was received, and generate an Ethernet priority-based flow control (PFC) frame that includes the one or more identified PCPs in a header of the Ethernet PFC frame. Additionally, the Ethernet gateway is configured to transmit the Ethernet PFC frame to an Ethernet fabric as a function of the one or more identified PCPs. Other embodiments are described herein.

Technologies for ethernet gateway congestion management in high-performance computing architectures

This disclosure describes enhancements to Ethernet for use in higher performance applications like Storage, HPC, and Ethernet based fabric interconnects. This disclosure provides various mechanisms for lossless fabric enhancements with error-detection and retransmissions to improve link reliability, frame pre-emption to allow higher priority traffic over lower priority traffic, virtual channel support for deadlock avoidance by enhancing Class of service functionality defined in IEEE 802.1Q, a new header format for efficient forwarding/routing in the fabric interconnect and header CRC for reliable cut-through forwarding in the fabric interconnect. The enhancements described herein, when added to standard and/or proprietary Ethernet protocols, broadens the applicability of Ethernet to newer usage models and fabric interconnects that are currently served by alternate fabric technologies like Infiniband, Fibre Channel and/or other proprietary technologies, etc.

Ethernet enhancements

Particular embodiments described herein provide for a network element that can be configured to receive a request message, wherein the request message includes a read trigger, an indicator selector, and a completion trigger, determine an indicator that relates to the indicator selector, and perform an action when the read trigger is activated.

Performance monitoring

Apparatuses, methods and storage medium associated with routing data in a switch are provided. In embodiments, the switch may include route lookup circuitry determine a first set of output ports that are available to send a data packet to a destination node. The lookup circuitry may further select, based on respective congestion levels associated with the first set of output ports, a plurality of output ports for a second set of output ports from the first set of output ports. An input queue of the switch may buffer the data packet and route information associated with the second set of output ports. The switch may further include route selection circuitry to select a destination output port from the second set of output ports, based on updated congestion levels associated with the output ports of the second set of output ports. Other embodiments may be described and/or claimed.

Apparatus and method for routing data in a switch

Technologies for synchronized sampling of counters include a computing device to determine a global clock to which the computing device and a plurality of other computing devices are to be synchronized. The computing device receives a request to sample a counter of the computing device from an administration server and records a state of the counter based on the global clock in response to receiving the request.

Technologies for synchronized sampling of counters based on a global clock

Apparatuses, methods and storage medium associated with buffering data in a switch are provided. In embodiments, the switch may include a plurality of queue buffers, a plurality of queues respectively associated with the plurality of queue buffers, a shared buffer, and a queue point controller coupled with the plurality of queue buffers and the shared buffer. In embodiments the queue point controller may be configured to determine an amount of available space in a selected queue buffer of the plurality of queue buffers. The queue point controller may be further configured to allocate at least a portion of the shared buffer to a selected queue that is associated with the selected queue buffer. In embodiments, this allocation may be based on the amount of available space determined in the selected queue buffer. Other embodiments may be described and/or claimed.

Apparatus and method for buffering data in a switch

Technologies for fabric security include one or more managed network devices coupled to one or more computing nodes via high-speed fabric links. A managed network device enables a port and, while enabling the port, securely determines the node type of the link partner coupled to the port. If the link partner is a computing node, management access is not allowed at the port. The managed network device may allow management access at certain predefined ports, which may be connected to one of more management nodes. Management access may be allowed for additional ports in response to management messages received from the management nodes. The managed network device may check and verify data packet headers received from a compute node at each port. The managed network device may rate-limit management messages received from a compute node at each port. Other embodiments are described and claimed.

Technologies for high-performance network fabric security

This disclosure describes enhancements to Ethernet for use in higher performance applications like Storage, HPC, and Ethernet based fabric interconnects. This disclosure provides various mechanisms for lossless fabric enhancements with error-detection and retransmissions to improve link reliability, frame pre-emption to allow higher priority traffic over lower priority traffic, virtual channel support for deadlock avoidance by enhancing Class of service functionality defined in IEEE 802.1Q, a new header format for efficient forwarding/routing in the fabric interconnect and header CRC for reliable cut-through forwarding in the fabric interconnect. The enhancements described herein, when added to standard and/or proprietary Ethernet protocols, broadens the applicability of Ethernet to newer usage models and fabric interconnects that are currently served by alternate fabric technologies like Infiniband, Fiber Channel and/or other proprietary technologies, etc.

Methods and apparatus for implementing time synchronization across exascale fabrics. A master clock node is coupled to a plurality of slave nodes via a fabric comprising a plurality of fabric switches and a plurality of fabric links, wherein each slave node is connected to the master clock node via a respective clock tree path that traverses at least one fabric switch. The fabric switches are configured to selectively forward master clock time data internally along paths with fixed latencies that bypass the switches' buffers and switch circuitry, which enables the entire clock tree paths to also have fixed latencies. The fixed latency of the clock tree path is determined for each slave node. The local clocks of the slave nodes are then synchronized with the master clock by using master clock time data received by each slave node and the fixed latency of the clock tree path from the master clock node to the slave node that is determined. Techniques for determining a clock rate mismatch between the master clock and a local clock is also provided.

Exascale fabric time synchronization

Apparatuses, methods and storage media associated with multiple multi-drop buses in a switch are provided herein. In some embodiments, the switch may include a multi-drop row bus to transmit a plurality of frames in a row dimension of the matrix switch and a multi-drop column bus to transmit the plurality of frames in a column dimension of the matrix switch. The switch may further include an input port to receive the plurality of frames and an output port to output the plurality of frames from the matrix switch. Other embodiments may be described and/or claimed.

Apparatus and method for using multiple multi-drop buses

Method, apparatus, and systems for reliably transferring Ethernet packet data over a link layer and facilitating fabric-to-Ethernet and Ethernet-to-fabric gateway operations at matching wire speed and packet data rate. Ethernet header and payload data is extracted from Ethernet frames received at the gateway and encapsulated in fabric packets to be forwarded to a fabric endpoint hosting an entity to which the Ethernet packet is addressed. The fabric packets are divided into flits, which are bundled in groups to form link packets that are transferred over the fabric at the Link layer using a reliable transmission scheme employing implicit ACKnowledgements. At the endpoint, the fabric packet is regenerated, and the Ethernet packet data is de-encapsulated. The Ethernet frames received from and transmitted to an Ethernet network are encoded using 64b/66b encoding, having an overhead-to-data bit ratio of 1:32. Meanwhile, the link packets have the same ratio, including one overhead bit per flit and a 14-bit CRC plus a 2-bit credit return field or sideband used for credit-based flow control.

Reliable transport of ethernet packet data with wire-speed and packet data rate match

Methods, apparatus, and systems for implementing hierarchical and lossless packet preemption and interleaving to reduce latency jitter in flow-controller packet-based networks. Fabric packets are divided into a plurality of data units, with data units for different fabric packets buffered in separate buffers. Data units are pulled from the buffers and added to a transmit stream in which groups of data units are interleaved. Upon receipt by a receiver, the groups of data units are separated out and buffered in separate buffers under which data units for the same fabric packets are grouped together. In one aspect, each buffer is associated with a respective virtual lane (VL), and the fabric packets are effectively transferred over fabric links using virtual lanes. VLs may have different levels of priority under which data units for fabric packets in higher-priority VLs may preempt fabric packets in lower-priority VLs. By transferring data units rather than entire packets, transmission of a packet can be temporarily paused in favor of a higher-priority packet. Multiple levels of preemption and interleaving in a nested manner are supported.

Hierarchical/lossless packet preemption to reduce latency jitter in flow-controlled packet-based networks

Growing community of inventors

Portland, OR, United States of America

Thomas D Lovett