Seagate Technology Incorporated

Emulex Design & Manufacturing Corporation

Emulex Corporation

Marc Timothy Jones

Jason Matthew Feist

Steven Scott Williams

Robert William Dixon

Praveen Viraraghavan

Hemant Vitthalrao Mane

Curtis Edward Nottberg

David Jerome Allen

Ernest John Frey

Jason Wayne Kinsey

Thomas V Spencer

Carl Joseph Mies

Murthy Kompella

Niranjan Anant Pol

Benjamin J Scott

Kevin Dale Bowman

Samuel Burk Siewert

Thomas Patrick Jackson

David Robert Wiley

Sammy Dwayne Sawyer

The technology disclosed herein pertains to a system and method for scaling storage using peer-to-peer NVMe communication, the method including selecting one of a plurality of NVMe devices as principal device to control communication with a host via a PCI switch, designating remainder of the plurality of NVMe devices as subordinate devices, and controlling the communication between the host and the subordinate devices using a PCI P2P DMA between the principal device and the subordinate devices.

Scalable storage using NVMe communication

A distributed data storage system can be configured with a host connected to a device and a distribution module. The distribution module identifies a manufacturing origin of the device and diverts a system operation from an upstream component connected to the distribution module to the device in response to the identified manufacturing origin of the device. The manufacturer installed operating parameters of the device are then used to complete the system operation.

Data storage system with distributed operations for known hardware

A distributed data storage system may connect a host to a device and a dedication module. The dedication module can identify a manufacturing origin of the device before dedicating a type of host-generated data access request to the device. The type of host-generated data access request is chosen by the dedication module in response to an operating capability correlated to the device by the dedication module based on the identified manufacturing origin. At least one host-generated request of the dedicated type is then completed with the device.

Distributed data storage system with dedicated processing lane

Apparatus and method for executing hidden computational functions in a distributed data processing environment. In some embodiments, a trust boundary includes a target device such as a storage device, and a source device such as a client device in a computer network. A storage device processor executes a hidden command function (HCF) routine to accumulate HCF output data in a local cache responsive to an HCF command received from the client device over a data interface. The processor further establishes a smaller retention boundary within the trust boundary that includes the storage device and excludes the client device. The HCF output data are stored locally in a non-volatile memory (NVM) of the storage device while not transferring any portion of the HCF output data outside the retention boundary, including to the client device. The HCF routine can update a block-chain ledger or take some other form to provide data security.

Performing remote hidden compute functions in one or more processing devices

A data storage system employing distributed memories can have at least one host connected to a plurality of data storage devices via a network controller. One or more performance bottlenecks through the network controller may be identified with a performance module. A peer group consisting of at least two of the plurality of data storage devices is created with the performance module in response to the identified performance bottleneck so that a task can be assigned by the performance module to the peer group. The task may be chosen to mitigate the performance bottleneck by avoiding involvement of the network controller in the task.

Distributed data storage system with bottleneck mitigation

A distributed data storage system can be configured with a host connected to a first device and an initialization module that performs a default initialization procedure on the first device in response to detecting the first device has an unknown manufacturing origin. Conducting the default initialization procedure may allow the first device to service data access requests from the host. In response to the connection of a second device to the host and initialization module, a manufacturing origin of the second device is identified with the initialization module before the initialization module customizes the default initialization procedure to a custom procedure that is executed to allow the second device to satisfy a background operation prescribed by the initialization module.

Custom initialization in a distributed data storage system

A data storage system may connect a host to a device and a hardware module that are utilized to satisfy at least one host-generated data access request to the device. A portion of the device is set as dormant by the hardware module prior to identifying an origin of the device with the hardware module during the satisfaction of the at least one host-generated data access request. In response to the identified origin, the previously dormant portion of the device is activated and subsequently utilized to execute a task assigned by the hardware module.

Distributed data storage system with dormant hardware activation

Apparatus and method for maintaining real-time coherency between a local cache of a target device and a client cache of a source device during execution of a distributed computational function. In some embodiments, a source device, such as a host computer, is coupled via a network interface to a target device, such as a data storage device. A storage compute function (SCF) command is transferred from the source device to the target device. A local cache of the target device accumulates output data during the execution of an associated SCF over an execution time interval. Real-time coherency is maintained between the contents of the local cache and a client cache of the source device, so that the client cache retains continuously updated copies of the contents of the local cache during execution of the SCF. The coherency can be carried out on a time-based granularity or an operational granularity.

Maintaining real-time cache coherency during distributed computational functions

Methods, apparatuses, and computer-readable media for streaming arbitrarily large amounts of data through computational storage programs of a computational storage device. A computational storage device comprises a storage media, a computational storage processor, and a controller. A firmware of the controller comprises a plurality of streaming drivers, each associated with a data source or data destination of the storage device. The firmware further comprises a buffer abstraction layer operable to read data from a data source through an associated ingress streaming driver of the plurality of streaming drivers to provide a source data stream for a computational storage program executing on the computational storage processor. The buffer abstraction layer is further operable to receive a destination data stream from the computational storage program and write data to a data destination through an associated egress streaming driver of the plurality of streaming drivers.

Data streaming for computational storage

A system includes a storage device and a computational storage processor. The storage device includes media. The computational storage processor is configured to, after issuance of a single command from a host device, receive data corresponding to the command, process the data as the data is received using a filter program and provide results data from the processed data.

In-line data flow for computational storage

A network device comprising a first attach point, a second attach point, a switch and persistent connection logic is provided. The first attach point may connect the network device to a first link, and the second attach point may connect the network device to a second link. The switch may connect the first attach point to the second attach point. The persistent connection logic may create a persistent connection between a first network element and a second network element, where the persistent connection comprises the network device, the first link and the second link. The network device may also implement a non-persistent connection between two network elements, where the non-persistent connection may comprises the network device.

SAS expander based persistent connections

Embodiments of the present invention provide for creating and using persistent connections in SAS networks. A persistent connection may be a connection that persists for longer than the usual SAS connection. More specifically, it is a connection that is not subject to periodic tear downs by SAS devices according to existing SAS protocols (such as, by using CLOSE or BREAK primitives). Instead, persistent connections may be removable by a link reset. Persistent connections may be used in situations in which the overhead associated with the usual tear down and re-establishment of connections in a SAS network may be considered too high in comparison with its intended benefits. Persistent connections may also be used to provide virtual direct attachment between two different SAS connected devices or between a SAS connected device and an expander.

Described herein is an improved mechanism for bridging between SAS and SATA drives based upon existing SAS expanders in a SAS domain. In particular, a bridge capable of translating between SAS and SATA protocols is embedded in or coupled to an expander. When a SAS initiator request is received at the expander, the expander can route the request, based on a routing table, either directly to a destination SAS device or to the bridge for necessary translation before it is transmitted to a destination SATA drive. The routing table includes corresponding relationships between all SAS addresses and Phys through which those SAS and SATA devices are attached to the expander. SATA devices can be virtualized in the expander through a few assigned addresses in the routing table in a SAS discovery process.

SAS expander-based SAS/SATA bridging

The use of loops in SAS networks is enabled by designating ports connected to loop connections as table loop ports (TLPs). Under normal operating conditions, each TLP is blocked from receiving BCNs, appears to the expander to have nothing connected to it, and is made invisible to initiators. The loop connection and TLPs may be enabled and used to access devices when a problem is detected. In particular, the TLP will now appear in a list of destination ports within the expander to which a BCN should be propagated. In addition, during a subsequent self-configuration, the TLP is allowed to populate its route table with devices accessible through it, and the existence of the TLP is also reported back to initiators. After re-discovery is complete, communications between the initiator and a target can resume, with traffic re-routed through the TLPs as needed, bypassing the failure point.

Enabling SAS expander loops for increased fairness and performance

Methods and systems for simplification of the re-discovery process for initiators due to changes in the network. If an initiator subscribes to change reports from a SAS expander, when that SAS expander detects a change in the network, it sends an SMP command back to the initiator, indicating the specific change in the network. Initiator BCN management and re-discovery of the entire network is therefore avoided.

Growing community of inventors

Costa Mesa, CA, United States of America

Marc Timothy Jones