Emc IP Holding Company LLC

Emc Corporation

Dell Products Inc.

Alesia Tringale

Jason J Duquette

Ian Wigmore

Julie Zhivich

Steven T McClure

Jerome Cartmell

Owen Martin

Arieh Don

James Marriot Guyer

Gabriel Benhanokh

Jeremy O'Hare

Zhi-Gang Liu

Yao Wang

Ken Dorman

Sean Pollard

Reema Gupta

Kuolin Hua

Anoop Raghunathan

Seema G Pai

Aidan O Mahony

Abhinav Garg

Adwait M Sathe

Philip O'Carroll

William Baxter

Lionel Bella

Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

Techniques for determining compression tiers and using collected compression hints

A storage system may assign a different encryption key to each logical storage unit (LSU) of a storage system. For each LSU, the encryption key of the LSU may be shared only with host systems authorized to access data of the LSU. In response to a read request for a data portion received from a host application executing on the host system, encryption metadata for the data portion may be accessed. If it is determined from the encryption metadata that the data portion is encrypted, the data encryption metadata for the data portion may be further analyzed to determine the encryption key for the data portion. The data may be retrieved from the storage system, for example, by performance of a direct read operation. The retrieved data may be decrypted, and the decrypted data may be returned to the requesting application.

Data encryption for directly connected host

Secure access to data on a storage system via direct connection to an internal fabric of the storage system may be provided. A storage system interface (SSI) may validate each I/O communication originating on the host system before allowing a corresponding I/O communication to be transmitted on the internal fabric. The validation may include applying predefined rules and/or ensuring that the I/O communication conforms to one or more technologies, e.g., NVMe. The SSI may be configured to encrypt I/O communications originating on a host system and to decrypt I/O communications received from the storage system, for example, in embodiments in which data is encrypted in flight from the host system to physical storage devices, and data may be encrypted at rest in memory of the storage system and/or on physical storage devices.

Secure host access to storage system resources via storage system interface and internal switching fabric

A hardware offload includes a hash engine that performs hashing for a block-based storage system. The hash engine calculates multiple hash values for each input buffer provided by the storage system. The hash values may be calculated with variably offset and overlapping portions of the input buffer, wherein each portion is larger than the native block size of the storage system. The hardware offload may also include a compression engine that performs compression on the input buffer using the entire input buffer and/or chunks as compression domains.

Multiple overlapping hashes at variable offset in a hardware offload

Data compression is performed on a storage system for which one or more host systems have direct access to data on the storage system. The storage system may compress the data for one or more logical storage units (LSUs) having data stored thereon, and may update compression metadata associated with the LSUs and/or the data portions thereof to reflect that the data is compressed. In response to a read request for a data portion received from a host application executing on the host system, compression metadata for the data portion may be accessed. If it is determined from the compression metadata that the data portion is compressed, the data compression metadata for the data portion may be further analyzed to determine how to decompress the data portion. The data portion may be retrieved and decompressed, and the decompressed data may be returned to the requesting application.

Data compression for having one direct connection between host and port of storage system via internal fabric interface

A host system is connected to the internal fabric of a storage system without an intervening external network or director or other component of the storage system controlling the host system's access to the internal fabric. The host system may exchange I/O communications with physical storage devices and/or global memory over an I/O path that does not include any directors, for example, over the internal fabric to which the host system is directly attached. In embodiments in which at least a portion of the global memory is considered part of a director, the host system may be configured to communicate with such global memory over the internal fabric and without use of director compute resources.

Host system directly connected to internal switching fabric of storage system

A storage system includes four storage engines, each storage engine including two compute nodes. Eight point-to-point connections are used to interconnect pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system. Atomic operations can be initiated by any compute node on any other compute node. Atomic operations received by a compute node on one of the point-to-point connections will be forwarded on the other point-to-point connection if the atomic operation is not directed to the compute node. During normal operation, atomic operations on a given compute node are performed on a host adapter associated with the compute node. Upon failure of the host adapter associated with the compute node, atomic operations may be performed on the compute node using the host adapter of the other compute node of the storage engine.

Storage system having storage engines with multi-initiator host adapter and fabric chaining

A storage system includes a storage engine having a first compute node, a second compute node, a first fabric adapter, and a second fabric adapter, the first compute node having a first memory and the second compute node having a second memory. The first compute node is connected to both the first and second fabric adapters, and the second compute node is connected to both the second and first fabric adapters. Both fabric adapters are configured to perform atomic operations on a memory of its respective compute node, and each fabric adapter contains a multi-initiating module configured to enable both the first compute node and the second compute node to initiate memory access operations on its respective memory.

Storage engine having compute nodes with redundant fabric access

A host system may include metadata mapping logical storage devices and logical addresses therein to physical storage devices and physical addresses therein. For a read operation, the host system, if it is determined that the data is not in cache on the storage system, the host system may determine, from the device-mapping metadata, the physical storage device and physical location (e.g., address range) therein of the data to be read. The data then may be read from the physical storage device over the internal fabric of the storage system without use of a director. Data may be read from the physical storage device to the host system using RDMA communications that do not involve use of any CPU resources on the host system or the storage system.

Host mapping logical storage devices to physical storage devices

A host system performs I/O processing functions traditionally performed on storage systems. Metadata about data stored on the storage system may be stored on the host system, including metadata about the data stored in a cache of the storage system. The SSI may be configured to determine whether an I/O operation is a read or write operation. If the I/O operation is a read operation, the SSI may determine from metadata stored thereon whether the data to be read is in cache. If the data is in cache, the SSI may read the data directly from cache over the internal fabric without use of CPU resources of a director, and, in some embodiments, without use of a director at all. If the data is not in cache, the SSI may read the data directly from the physical storage device over the internal fabric without use of a director.

Host processing of I/O operations

A storage system interface (SSI) located externally to a data storage system serves as an interface between a host system and the data storage system. The SSI may be part of the host system, and in some embodiments may be a separate and discrete component from the remainder of the host system, physically connected to the remainder of the host system by one or more buses that connect periphery devices to the remainder of the host system. The SSI may be physically connected directly to the internal fabric of the data storage system, and may be implemented on a card or chipset physically connected to the remainder of a host system by a PCIe bus. The SSI may provide functionality traditionally provided on data storage systems, enabling at least some I/O processing to be offloaded from data storage systems to hosts that include SSIs.

Intelligent external storage system interface

A method, computer program product, and computer system for identifying, by a computing device, a trigger event associated with an Ethernet switch. Ethernet based control information may be encapsulated into an InfiniBand based packet. The InfiniBand based packet with the Ethernet based control information may be transmitted over the InfiniBand fabric from a source to a destination. The Ethernet based control information may be decapsulated from the InfiniBand based packet at the destination.

System and method for high availability of ethernet network during storage system failure

Compressing data of a storage device includes selecting a portion of data for data compression that is predicted to be unlikely to be accessed, selecting a particular one of a plurality of data compression algorithms to apply to the portion according to a frequency value associated with each of the data compression algorithms, and adjusting the frequency value of the particular one of a plurality of data compression algorithms according to performance of the particular one of a plurality of data compression algorithms. The performance may vary according to a ratio of amount of compression achieved to processing cycles of a processor used to compress the portion of data. The processor may perform storage device functions that are separate from data compression. The portion of data may be selected from a plurality of logical devices used in connection with a single application that accesses the storage device.

Accelerating data reduction through reinforcement learning

A parallel decompression engine has separate hardware paths for sequence processing and repeated byte copying/placement. Literal byte extraction logic obtains literal bytes from a selected sequence. Literal byte write logic writes the obtained literal bytes into an uncompressed data set that is being generated. Offset and length extraction logic obtains the offset and length of repeated bytes from the selected sequence. In a separate hardware path, copying and placement logic uses the offset and length to find and copy the length of repeated bytes at the specified offset in the uncompressed data set, and place the copied repeated bytes back into the uncompressed data set adjacent to the literal bytes.

Growing community of inventors

Worcester, MA, United States of America

Alesia Tringale