Advanced Micro Devices Corporation

Joseph A Bailey

Norman M Hack

Joseph Winkles

Qadeer Ahmad Qureshi

Dan S Mudgett

Bahadir Erimli

Rodney W Schmidt

Steve L Belt

Clark L Buxton

A channel adapter, configured for sending frame data according to link widths selected based on management frames received from a link partner, includes a multiplexer circuit configured for selectively switching the frame data supplied according a prescribed maximum link width, to one of a plurality of available link widths for a transmit bus, and a bus controller. The bus controller is configured for controlling the multiplexer circuit to switch the frame data to one of the available link widths, including the prescribed link width, based the selected link width. Hence, a channel adapter can be configured for an optimum link width for communication with a corresponding channel adapter on a peer node.

Arrangement in a channel adapter for transmitting data according to link widths selected based on received link management packets

A host channel adapter includes a transport layer module, a link layer module, and buffer memory having memory portions configured for storage of transmit data packets output by the transport layer module for transmission by the link layer module on identified virtual lanes. The transport layer module is configured for identifying a virtual lane for each transmit data packet, and for storing the transmit data packet in the corresponding memory portion assigned to the corresponding identified virtual lane. Hence, the transmit data packets output by the transport layer module are stored in the memory portions based on their respective identified virtual lanes, where each memory portion stores the transmit data packets for the corresponding identified virtual lane. The link layer module retrieves the transmit data packets from a selected memory portion corresponding to a currently-serviced virtual lane based on a prescribed virtual lane arbitration. Hence, the link layer module can retrieve the transmit data packets for the currently-serviced virtual lane with minimal processing, based on accessing the corresponding memory portion.

Arrangement in a channel adapter for segregating transmit packet data in transmit buffers based on respective virtual lanes

An host channel adapter embedded within a processor device includes a transport layer module, a transport layer buffer, a link layer module, and a link layer buffer configured for storing at least two packets to be transmitted by the embedded host channel adapter. The transport layer module is configured for generating, for each packet to be transmitted, a transport layer header, and storing in the transport layer buffer the transport layer header and a corresponding identifier that specifies a stored location of a payload for the transport layer header. The link layer module includes payload fetch logic configured for fetching the payload based on the corresponding identifier, enabling the link layer module to construct one of the two packets to be transmitted concurrently during transmission of the second of the two packets.

Embedded channel adapter having link layer configured for concurrent retrieval of payload data during packet transmission

A channel adapter includes a link receive resource configured for initiating packet validation upon detecting a receive counter reaching a prescribed threshold corresponding to reception of an initial header of a data packet. Upon initiating packet validation, the link receive resource determines whether the initial header includes any errors. Any errors detected in the initial header are stored if the errors are detected prior to reception of an end of the data packet. Additional validation operations can be initiated upon reception of the respective headers. Upon receiving the end of the data packet, the link receive resource selectively reports the errors detected in the initial header based on whether any higher-priority error is detected relative to a prescribed error reporting order.

Arrangement in a channel adapter for validating headers concurrently during reception of a packet for minimal validation latency

A host channel adapter is configured for servicing received work notifications based on identifying the work notifications associated with the virtual lanes (VL) having a prescribed ordering position identified by the link layer operations. The host channel adapter, in response to receiving a work notification for a prescribed service level (SL), determines the virtual lane associated with the specified service level based on a prescribed service level to virtual lane mapping. If necessary (e.g., for an unreliable datagram service type), the work notification supplies the prescribed service level (SL) for the host channel adapter. The host channel adapter also determines an ordering position for the determined virtual lane from the link layer module, and selectively services the work notification based on the corresponding ordering position.

Arrangement in a channel adapter for servicing work notifications based on link layer virtual lane processing

An embedded host channel adapter includes a transport layer module, a transport layer buffer, and a link layer module. The transport layer buffer is configured for storing transmit packet entries for virtual lanes serviced by the embedded host channel adapter. The link layer module is configured for supplying virtual lane priority information and virtual lane flow control information, for each virtual lane, to the transport layer module. The link layer module also configured for constructing transmit packets to be transmitted based on retrieval thereof from the transport layer buffer. The transport layer module is configured for selecting one of the virtual lanes for servicing based on the supplied virtual lane priority information and virtual lane flow control information for each of the virtual lanes, enabling the transport layer module to prioritize received work notifications, for generation of respective transmit packet entries.

Embedded channel adapter having transport layer configured for prioritizing selection of work descriptors based on respective virtual lane priorities

A consumer resource provider is configured for generating a work request to a prescribed virtual destination address on behalf of a user-mode consumer process requiring a memory access. An operating system resource, configured for establishing communications between the consumer resource provider and a host channel adapter configured for servicing the work notifications, assigns virtual address space for use by the consumer resource provider, and respective unique mapping values specified as user mode access for use by the consumer resource provider in executing the memory accesses on behalf of the respective user-mode consumer processes. An address translator includes a translation map for uniquely mapping the virtual address space used by the consumer resource provider to a prescribed physical address space accessible by the host channel adapter. The address translator, in response to receiving the work notification at a virtual address from the consumer resource provider on behalf of an identified user-mode consumer process, maps the work notification to a corresponding prescribed physical address based on the corresponding mapping value assigned for the identified user-mode consumer process, enabling the host channel adapter to detect the work notification at the mapped physical address.

Arrangement for implementing kernel bypass for access by user mode consumer processes to a channel adapter based on virtual address mapping

A method and apparatus for reordering transactions in a packet-based fabric using I/O Streams. Packet bus transactions may flow upstream from node to node on a non-coherent I/O packet bus. Some peripheral buses place ordering constraints on their bus transactions to prevent deadlock situations. When a packet transaction originating on a peripheral bus with ordering constraints is translated to a packet bus such as the non-coherent I/O packet bus, those same ordering constraints may be mapped over to the packet bus transactions. To efficiently handle the packets and prevent deadlock situations, packets may be handled and reordered on an I/O stream basis. Thus, reordering logic may consider I/O streams independently and therefore only reorder transactions within an I/O stream and not across more than one I/O stream.

Method and apparatus for reordering transactions in a packet-based fabric using I/O streams

A computing node configured for communications on an InfiniBand&#x2122; network includes at least two host channel adapters configured for communications on the InfiniBand&#x2122; network, and at least one processor configured for controlling the communications of the two host channel adapters on the InfiniBand&#x2122; network. The host channel adapters communicate with the processor via an internal bus. The processor monitors communication operations by the host channel adapters on the InfiniBand&#x2122; network. If the processor detects that one of the host channel adapters is unable to complete the corresponding communication operations, the processor outputs a message requesting traffic destined to the one host channel adapter to be redirected to the remaining host channel adapter.

Fault tolerant computing node having multiple host channel adapters

A method and apparatus for manipulating work queue elements via a hardware adapter and a software driver. The software driver is configured to cause a plurality of work queue elements to be stored in a queue pair including a plurality of storage locations. Each of the plurality of storage locations includes an indicator indicating whether a corresponding work queue element has been completed. The hardware adapter is configured to select one of the plurality of storage locations and to service a corresponding one of the plurality of work queue elements, and in response to completion of a task associated with the corresponding work queue element, to cause the indicator to indicate that the corresponding work queue element has been completed. Additionally, the software driver is configured to cause a new work queue element to be stored in the selected storage location in response to detecting that the indicator indicates that the corresponding work queue element has been completed.

Manipulating work queue elements via a hardware adapter and software driver

A host channel adapter is configured for servicing a work notification, supplied by a host process to an assigned destination address accessable by the host channel adapter, based on matching the assigned destination address with a stored notification address from one of a plurality of queue pair context entries stored within the host channel adapter. The host channel adapter receives a queue pair context entry including a notification address, based on creation of a corresponding queue pair for a host process. The queue pair enables the host process to post a work descriptor and output a work notification to the host channel adapter by writing the work notification to an assigned destination address. The host channel adapter matches the assigned destination address with a stored notification address, and services the work descriptor based on the corresponding queue pair attributes specified in the identified queue pair context entry.

Arrangement for instigating work in a channel adapter based on received address information and stored context information

An operating system resource, configured for establishing communications between consumer processes configured for generating respective work notifications and a host channel adapter configured for servicing the work notifications, assigns virtual address space for use by the consumer processes in executing memory accesses, and respective unique mapping values. An address translator includes a translation map for uniquely mapping the virtual address space used by the consumer processes to a prescribed physical address space accessible by the host channel adapter. The address translator, in response to receiving from an identified consumer process the work notification at a virtual address, maps the work notification to a corresponding prescribed physical address based on the corresponding mapping value assigned to the identified consumer process, enabling the host channel adapter to detect the work notification for the consumer process.

Arrangement for limiting access to addresses by a consumer process instigating work in a channel adapter based on virtual address mapping

An interrupt messaging scheme to manage interrupts within a multiprocessing computer system without a dedicated interrupt bus. An interconnect structure using a plurality of high-speed dual-unidirectional links to interconnect processing nodes, I/O devices or I/O bridges in the multiprocessing system is implemented. Interrupt messages are transferred as discrete binary packets over point-to-point unidirectional links. A suitable routing algorithm may be employed to route various interrupt packets within the system. Simultaneous transmission of interrupt messages from two or more processing nodes and I/O bridges may be possible without any need for bus arbitration. Interrupt packets carry routing and destination information to identify source and destination processing nodes for interrupt delivery. A lowest priority interrupt packet from an I/O bridge is converted into a coherent form by the host processing node coupled to the I/O bridge. The host node then broadcasts the coherent interrupt packet to all other processing nodes in the system regardless of whether a processing node is identified as a target in the interrupt message packet from the I/O bridge. The host node also receives responses from recipient nodes and coordinates lowest priority arbitration. In case of a fixed, an ExtINT or a non-vectored interrupt message from the I/O bridge, the host node simply forwards the interrupt packets to all other nodes in the system without performing the conversion. Inter-processor interrupts may also be delivered in a similar manner. Interrupt response is decoupled from corresponding interrupt message and the interrupt messaging protocol may be implemented independently of the physical properties of a system bus carrying interrupt packets.

Point-to-point interrupt messaging within a multiprocessing computer system

An interrupt messaging scheme for a multiprocessing computer system where a dedicated bus to carry interrupt messages within the multiprocessing system is eliminated. Instead, an interconnect structure using a plurality of high-speed dual-unidirectional links to interconnect processing nodes, I/O devices or I/O bridges in the system is implemented. Interrupt messages are transferred as discrete binary packets over point-to-point unidirectional links. Various interrupt requests are transferred through a predetermined set of discrete interrupt message packets. Interrupt message initiators&mdash;an I/O interrupt controller or a local interrupt controller (in case of an inter-processor interrupt)&mdash;may be configured to generate appropriate interrupt message packets upon receiving an interrupt request. A suitable routing algorithm may be employed to route various interrupt messages within the system. Simultaneous transmission of interrupt messages from two or more interrupt controllers may be possible without any need for bus arbitration. Interrupt response is decoupled from corresponding interrupt message and the interrupt messaging protocol may be implemented independently of the physical properties of a system bus carrying interrupt packets. System flexibility in managing interrupts is thus improved.

Method for distributing interrupts in a multi-processor system

A programmable interrupt controller for use in computer systems including one or more CPUs is provided. The programmable interrupt controller includes an interrupt request interface, a central interrupt controller, random access memory, and at least one processor interface. The central interrupt controller systematically selects interrupt requests from the interrupt request interface. Information associated with each interrupt request is stored in the random access memory. The central interrupt controller access the information in the random access memory and uses the information and the state of the currently selected interrupt request to determine a next state for the currently selected interrupt request. The information is passed on to the processor interface to determine when and if the interrupt request should issue to one of the CPUs.

Growing community of inventors

Austin, TX, United States of America

Joseph A Bailey