Intel Corporation

Cray Research, Inc.

Cray Incorporated

Silicon Graphics, Incorporated

Mark S Birrittella

Thomas D Lovett

Todd M Rimmer

Randal Steven Passint

Steven M Oberlin

Richard E Kessler

Greg Thorson

Jan A Wikstrom

Mahesh U Wagh

Venkatraman Iyer

Zuoguo Wu

Gerald S Pasdast

Su Wei Lim

Michael A Parker

Brent Ryan Rothermel

Eric Carl Fromm

Albert S Cheng

James Kunz

Paul T Wildes

Vernon W Swanson

Lip Khoon Teh

Robert J Lutz

David R Kiefer

Stephen B Smetana

Anoop Kumar Upadhyay

Harro Zimmermann

James A Kunz

Various embodiments are generally directed to dynamically changing a width of a network link without inactivating the network link. Provided are component communicatively coupled via a network link configured to negotiate a new network link width and to initiate a soft recovery of the network link to implement the new negotiated link width.

Coordinating width changes for an active network link

Methods and apparatus for implementing time synchronization across exascale fabrics. A master clock node is coupled to a plurality of slave nodes via a fabric comprising a plurality of fabric switches and a plurality of fabric links, wherein each slave node is connected to the master clock node via a respective clock tree path that traverses at least one fabric switch. The fabric switches are configured to selectively forward master clock time data internally along paths with fixed latencies that bypass the switches' buffers and switch circuitry, which enables the entire clock tree paths to also have fixed latencies. The fixed latency of the clock tree path is determined for each slave node. The local clocks of the slave nodes are then synchronized with the master clock by using master clock time data received by each slave node and the fixed latency of the clock tree path from the master clock node to the slave node that is determined. Techniques for determining a clock rate mismatch between the master clock and a local clock is also provided.

Exascale fabric time synchronization

Method, apparatus, and systems for reliably transferring Ethernet packet data over a link layer and facilitating fabric-to-Ethernet and Ethernet-to-fabric gateway operations at matching wire speed and packet data rate. Ethernet header and payload data is extracted from Ethernet frames received at the gateway and encapsulated in fabric packets to be forwarded to a fabric endpoint hosting an entity to which the Ethernet packet is addressed. The fabric packets are divided into flits, which are bundled in groups to form link packets that are transferred over the fabric at the Link layer using a reliable transmission scheme employing implicit ACKnowledgements. At the endpoint, the fabric packet is regenerated, and the Ethernet packet data is de-encapsulated. The Ethernet frames received from and transmitted to an Ethernet network are encoded using 64b/66b encoding, having an overhead-to-data bit ratio of 1:32. Meanwhile, the link packets have the same ratio, including one overhead bit per flit and a 14-bit CRC plus a 2-bit credit return field or sideband used for credit-based flow control.

Reliable transport of ethernet packet data with wire-speed and packet data rate match

Methods, apparatus, and systems for implementing hierarchical and lossless packet preemption and interleaving to reduce latency jitter in flow-controller packet-based networks. Fabric packets are divided into a plurality of data units, with data units for different fabric packets buffered in separate buffers. Data units are pulled from the buffers and added to a transmit stream in which groups of data units are interleaved. Upon receipt by a receiver, the groups of data units are separated out and buffered in separate buffers under which data units for the same fabric packets are grouped together. In one aspect, each buffer is associated with a respective virtual lane (VL), and the fabric packets are effectively transferred over fabric links using virtual lanes. VLs may have different levels of priority under which data units for fabric packets in higher-priority VLs may preempt fabric packets in lower-priority VLs. By transferring data units rather than entire packets, transmission of a packet can be temporarily paused in favor of a higher-priority packet. Multiple levels of preemption and interleaving in a nested manner are supported.

Hierarchical/lossless packet preemption to reduce latency jitter in flow-controlled packet-based networks

A system-on-a-chip, such as a logical PHY, may be divided into hard IP blocks with fixed routing, and soft IP blocks with flexible routing. Each hard IP block may provide a fixed number of lanes. Using p hard IP blocks, where each block provides n data lanes, h=n*p total hard IP data lanes are provided. Where the system design calls for k total data lanes, it is possible that k&#x2260;h, so that &#x250c;k/n&#x2510; hard IP blocks provide h=n*p available hard IP data lanes. In that case, h&#x2212;k lanes may be disabled. In cases where lane reversals occur, such as between hard IP and soft IP, bowtie routing may be avoided by the use of a multiplexer-like programmable switch within the soft IP.

Multichip package link

Method, apparatus, and systems for detecting lane errors and removing errant lanes in multi-lane links. Data comprising link packets is split into a plurality of bitstreams and transmitted over respective lanes of a multi-lane link in parallel. The bitstream data is received at multiple receive lanes of a receiver port and processed to reassemble link packets and to calculate a CRC over the data received on each lane. The link packets include a transmitted CRC that is compared to a received CRC to detect link packet errors. Upon detection of a link packet error, per-lane or per transfer group CRC values are stored, and a retry request is issued to retransmit the bad packet. In conjunction with receipt of the retransmitted packet, per-lane or per transfer group CRC values are recalculated over the received data and compared with the stored per-lane or per transfer group CRC values to detect the lane causing the link packet error.

Lane error detection and lane removal mechanism to reduce the probability of data corruption

Methods, apparatus, and systems for implementing a link layer retry protocol utilizing implicit ACKnowledgements (ACKs). Peer link interfaces are configured to facilitate confirmed error-free delivery of link-layer packets through use of implicit ACKs, while also providing retransmission of packets for which errors are detected and guaranteeing the link control data is either successfully received or data transfer over the link is prevented. In conjunction with transmitting packets, reliable packets are copied into sequential slots in a replay buffer. Each link interface tracks the slot at which each reliable packet is buffered, and in response to detection of an error, a retry request is sent to the transmit-side to retransmit the errant packet. The previously buffered copy of the errant packet is retrieved from the replay buffer and retransmitted. Through use of a link roundtrip detection mechanism, absence of a retry request by the time a replay buffer has returned to the slot of a reliable packet (plus a predetermined number of additional transfer cycles, if applicable) provides an implicit ACK that the packet was received without error.

Efficient link layer retry protocol utilizing implicit acknowledgements

Method, apparatus, and systems for Link Transfer, bit error detection and link retry using flit bundles asynchronous to link Fabric Packets. A first type of packet comprising a Fabric Packet is generated and its data content is divided into multiple data units called 'flits.' The flits are then bundled into a second type of packet comprising Link Transfer Packets (LTPs). The LTPs are then sent over single link segments in a fabric comprising many point-to-point links. Each LTP includes a CRC that is used to ensure that data transmitted over each link segment is error free, and comprises a unit of retransmission. The size of the fabric packets may vary, and they may be larger or smaller than an LTP. The transfer scheme enabled flits from multiple fabric packets to be bundled into a single LTP. Upon receipt at a fabric endpoint, the flits from the LTPs are extracted and reassembled to regenerate the Fabric Packets.

Link transfer, bit error detection and link retry using flit bundles asynchronous to link fabric packets

A method includes specifying a first set of interconnected devices associated with a first leaf cell in Verilog syntax, and specifying a second set of interconnected devices associated with a second leaf cell in Verilog syntax. A connection between the first leaf cell and the second leaf cell is also specified in Verilog syntax. This specifies a circuit. The functionality of the logic can be tested by running a logic simulation on the circuit without converting to Verilog syntax. The Verilog syntax, associated with the circuit, can be converted directly from Verilog syntax to a SPICE netlist. The SPICE netlist can be used to simulate the timing and other parameters of the circuit. The Verilog syntax can be used to verify the circuit. Also included are a computer readable medium including an instruction set for the above method, and a data structure necessary to carry out the above method.

Transistor level verilog

Systems and methods for independently adjusting a duty cycle of an input clock signal in an IC to compensate for uncertainties and distortions in the logic signals resulting from the logic signals propagating through the IC to improve system performance. This is accomplished by inputting first and second programming instructions into one of a plurality of edge-triggered circuits to select one of a series of plurality of incremental or decremental duty cycle adjust circuits to adjust the duty cycle of a clock signal as a function of the first and second programming instructions.

Clock signal duty cycle adjust circuit

Systems and methods for synchronizing a system clock signal with a reference clock signal having a reduced phased offset to improve operating speeds of integrated circuits. This is accomplished by generating delayed system and reference clock signals by using the system and reference clock signals. The generated delayed clock signals are then monitored to determine the arrival of the raising and falling edges of the delayed clock signals. The system clock signal is then compensated based on the determination of the arrival of the delayed clock signals to substantially synchronize the system clock signal with respect to the reference clock signal.

Systems and methods for phase detector circuit with reduced offset

The present invention provides a system for efficient, high speed, high bandwidth, digital communication where transmit distances are greater than a single clock period. The digital system operates based on a system clock. Within the digital system a transmit module transmits data along with a capture clock signal to a receive module where the transmission time between the modules is greater than one period of the system clock. The capture clock operates in a known relationship to the system clock at a frequency at least twice as slow as the system clock. The digital system also has a synchronizing clock that operates at the same frequency as the forwarded clock. When the data arrives at the receive module it is captured by a pair of memory devices operating on different phases of the capture clock. The memory devices feed the data to a multiplexor that selects, as a function of the synchronizing clock, between the outputs of the two memory devices. At this point the data has been synchronized with the system clock and can be captured using the system clock for processing in the receive module.

Growing community of inventors

Chippewa Falls, WI, United States of America

Mark S Birrittella