Sun Microsystems, Inc.

Oracle America, Inc.

Oracle International Corporation

Other

John E Watkins

Elisa Rodrigues

William C Van Loo

Joseph Rollin Wright

Paul Jeffrey Garnett

Shimon Muller

Rahoul Puri

Stephen Rowlinson

Denton E Gentry, Jr

Linda T Cheng

William A Shannon

Joseph F Moran

William Van Loo

David Labuda

Ray Cheng

Arvind Srinivasan

William N Joy

Ola Tørudbakken

John P Petry

John R Feehrer

Abbas Morshed

Marcelino M Dignum

Robert Barton Garner

Kenneth S Goss

Ravinandan R Buchamwandla

Aron J Silverton

Michelle L Wong

Babu Rao Kandimalla

Nimita Taneja

Arvind Srinivsan

William R Malion

Lance G Hartmann

Hugh Chang

Kurt Michels

Louise Y Yeung

A switched fabric hierarchy (e.g., a PCIe hierarchy) may utilize hardware, firmware, and/or software for filtering duplicative or otherwise undesirable correctable error messages from reaching a root complex. An operating system of the root complex may detect a persistent stream or storm of correctable errors from a particular endpoint and activate filtering of correctable errors from that endpoint. A filtering device may receive filtering commands and parameters from the operating system, implement the filtering, and monitor further correctable errors from the offending device. While an offending device is being filtered, correctable error messages from the offending device may be masked from the operating system, while correctable error messages from other devices in the switched fabric hierarchy may be transmitted. At such time as the filtering device may detect that conditions for ending filtering of a device are met, the filtering device may cease filtering of the offending device and return monitoring responsibilities to the operating system.

Correctable error filtering for input/output subsystem

A system that includes a switched fabric hierarchy (e.g., a PCIe hierarchy) may realize efficient utilization of a shared I/O device (e.g., a network or storage switch) across multiple physically separate processing nodes (endpoints). For example, each processing node (endpoint) in a distributed processing system may be allocated a portion of the address map of a shared I/O device and may host a device driver for one of multiple virtual functions implemented on the shared device. Following enumeration and initialization of the hierarchy by the root complex, the endpoints may access the virtual functions directly (without intervention by the root complex). Data and interrupt traffic between endpoints and virtual functions may take place over peer-to-peer connections. Interrupt reception logic in each endpoint may receive and handle interrupts generated by the virtual functions. The root complex may host a device driver for a physical function on the shared device.

Virtualized I/O device sharing within a distributed processing node system

A firmware controller of a node of a distributed system determines a self-regulation threshold for at least one metric associated with traffic transmitted via an interconnect. The threshold is set to a value lower than the maximum permitted by the interconnect protocol. The controller transmits a request to an application-layer traffic endpoint to limit traffic based on the threshold. A hardware congestion management unit collects measurements on traffic metrics, including at least one metric other than the one for which the threshold is defined. Based on measurements obtained from the hardware congestion management unit, the controller modifies the self-regulation threshold and notifies the application-layer endpoint.

Congestion management in distributed systems using autonomous self-regulation

A system that includes a PCIe hierarchy may utilize a ring controller for message handling. Nodes acting as the root complex or as endpoint devices may include such ring controllers, portions of which may be implemented by dedicated circuitry on each node. The ring controllers may receive posted transactions representing messages, may return flow control credits for those transactions, may classify each message as to its type, and may write information about each message to a respective ring buffer storing information about messages of that type. A processor (or processing logic/circuitry) on the node may subsequently retrieve messages from the ring buffers and process them. The sizes and locations of the ring buffers in memory may be configurable by software (e.g., by writing to registers within the ring controllers). The message types may include correctable and non-correctable error messages, and non-error messages (including, but not limited to, vendor-defined messages).

Ring controller for PCIe message handling

An I/O device includes a host interface configured to process function level reset (FLR) requests in a specified amount of time. The host interface includes a control unit and groups of configuration space registers, each group corresponding to a function. The host interface also includes application availability registers, each associated with a respective function, and which may indicate whether application hardware within the respective function is available for access by a corresponding application device driver. The I/O device also includes application hardware resources associated with a respective function. In response to receiving an FLR request of a particular function, the control unit may cause the associated application availability register to indicate that the application hardware within the particular function is not available to the driver. The control unit may reset the corresponding configuration space registers within a predetermined amount of time and reset the associated application hardware resources.

Input/output device including a host interface for processing function level reset requests and updating a timer value corresponding to a time until application hardware registers associated with the function level reset requests are available

A system includes a virtualized I/O device coupled to one or more processing units. The virtualized I/O device includes programmed I/O (PIO) configuration registers corresponding to hardware resources, and a storage for storing a resource table that includes a plurality of entries. Each entry corresponds to a respective hardware resource. A system processor may allocate the hardware resources to functions that may include physical and virtual functions, and may program each entry of the resource discovery table for each function with an encoded value that indicates whether a requested hardware resource has been allocated to a requesting process, and whether the requested hardware resource is shared with another function. Processing units may execute a device driver instance associated with a given process to discover allocated resources by requesting access to the resource discovery table. The virtualized I/O device protects the resources by checking access requests against the resource discovery table.

System and method for discovering and protecting shared allocated resources in a shared virtualized I/O device

An I/O device includes a host interface that may be configured to receive and process a plurality of transaction packets sent by a number of processing units, with each processing unit corresponding to a respective root complex. The host interface includes an error handling unit having error logic implemented in hardware that may be configured to determine whether each transaction packet has an error and to store information corresponding to any detected errors within a storage. More particularly, the error handling unit may perform the error detection and capture of the error information as the transaction packets are received, or in real time, while the error handling unit may include firmware that may subsequently process the information corresponding to the detected errors.

Input/output device including a mechanism for error handling in multiple processor and multi-function systems

An I/O device includes a host interface coupled to a plurality of hardware resources. The host interface includes a transaction layer packet (TLP) processing unit that may receive and process a plurality of transaction layer packets sent by a plurality of processing units. Each processing unit may correspond to a respective root complex. The TLP processing unit may identify a transaction type and a processing unit corresponding to each transaction layer packet and store each transaction layer packet within a storage according to the transaction type and the processing unit. The TLP processing unit may select one or more transaction layer packets from the storage for process scheduling based upon a set of fairness criteria using an arbitration scheme. The TLP processing unit may further select and dispatch transaction layer packets for processing by downstream application hardware based upon additional criteria.

Input/output device including a mechanism for transaction layer packet processing in multiple processor systems

A system includes a virtualized I/O device coupled to one or more processing units. The virtualized I/O device includes a storage for storing a resource discovery table, and programmed I/O (PIO) configuration registers corresponding to hardware resources. A system processor may allocate the plurality of hardware resources to one or more functions, and to populate each entry of the resource discovery table for each function. The processing units may execute one or more processes. Given processing units may further execute OS instructions to allocate space for an I/O mapping of a PIO configuration space in a system memory, and to assign a function to a respective process. Processing units may execute a device driver instance associated with a given process to discover allocated resources by requesting access to the resource discovery table. The virtualized I/O device protects the resources by checking access requests against the resource discovery table.

System and method for discovering and protecting allocated resources in a shared virtualized I/O device

An I/O device includes a host interface that may receive and process transaction packets sent by a number of processing units, with each processing unit corresponding to a respective root complex. The host interface includes an error handling unit having error logic implemented in hardware that may determine, as each packet is received, whether each transaction packet has an error and to store information corresponding to any detected errors. The error handling unit may include an error processor that may be configured to execute error processing instructions to determine any error processing operations based upon the information. The error processor may also generate and send one or more instruction operations, each corresponding to a particular error processing operation. The error handling unit may also include an error processing unit that may execute the one or more instruction operations to perform the particular error processing operations.

Input/output device including a mechanism for accelerated error handling in multiple processor and multi-function systems

A method for reducing address space in a shared virtualized I/O device includes allocating hardware resources including variable resources and permanent resources, to one or more functions. The method also includes allocating address space for an I/O mapping of the resources in a system memory, and assigning a respective portion of that address space for each function. The method further includes assigning space within each respective portion for variable resources available for allocation to the function to which the respective portion is assigned, and further assigning space within each respective portion for a set of permanent resources that have been allocated to the function to which the respective portion is assigned. The method further includes providing a translation table having a plurality of entries, and storing within each entry of the translation table, a different internal address of a permanent resource that has been allocated to a particular function.

Method and system for reducing address space for allocated resources in a shared virtualized I/O device

An I/O device having function level reset functionality includes a host interface that may include a master reset unit, a plurality of client interfaces, each corresponding to one or more functions, and a plurality of hardware resources. Each hardware resource may be associated with a respective function. In response to receiving a reset request to reset a specific function, the master reset unit may provide to each client interface, a request signal corresponding to the reset request, and a signal identifying the specific function. Each client interface having an association with the specific function may initiate a reset operation of the associated hardware resources, and also provide a client reset done signal for the specific function to the master reset unit in response to completion of the reset operations of the hardware resources. The master reset unit provides a reset done signal for the specific function to the host interface.

Mechanism for performing function level reset in an I/O device

The described embodiments provide a system for accessing values for configuration space registers (CSRs). This system includes a CSR data storage mechanism with an address input and a CSR data output. The CSR data storage mechanism includes a memory containing a number of memory locations for storing the true or actual values for CSRs for functions for corresponding devices. In these embodiments, the memory locations are divided into at least one shared region and at least one unique region. In these embodiments, in response to receiving an address for a memory location on the address input, the CSR data storage mechanism accesses the value for the CSR in the memory location in a corresponding shared region or unique region.

Configuration space compaction

A method and apparatus for recovering from errors occurring during system bus transactions. An input/output device such as a network interface unit (NIU) issues read and write operations across a meta interface coupling the device to host bus (glue) logic. The host bus logic translates the operations into system bus transactions. The device maintains a set of reusable identifiers for identifying the operations, and a table maintained by the device or the host bus logic maps the operation identifiers to transaction identifiers identifying the system bus transactions spawned to perform the operations. If a bus transaction encounters an unrecoverable error, the host bus logic reports the error to the device and drops any further data received from other bus transactions performed for the same operation. The device marks the operation's identifier as dirty, to prevent its reuse. The operation identifier may be reused after software clears the error condition.

Method and apparatus for recovering from system bus transaction errors

In an interconnect apparatus for interconnecting at least one host to at least a plurality of presentation registers provide a presentation interface for the device to the host. The interconnect apparatus includes memory for holding the presentation registers and a governor operable to manage the presentation registers in the memory.

Growing community of inventors

Sunnyvale, CA, United States of America

John E Watkins