Datadirect Networks, Inc.

Hitachi Global Storage Technologies Netherlands B.v.

Hgst Netherlands, B.v.

Citigroup Technology, Inc.

Hget Netherlands B.v.

Donald Joseph Molaro

Jorge Campello De Souza

Adam Fried-Gintis

Dan Olster

Damien C D Le Moal

Richard M H New

David F Fellinger

Michael Piszczek

Jason Micah Cope

Paul Joseph Nowoczynski

John Gordon Manning

Pavan Kumar Uppu

Chunqi Han

Stephen M Hopkins

Chris Howarth

Zvonimir Z Bandic

Marco Sanvido

Anand Kulkarni

Atsushi Kanamaru

Frank R Chu

Timothy George Harrington, Ii

Tadahisa Kawa

Frank Rui-Feng Chu

Chungi Han

Pavan Kumar Kumar Uppu

A data storage system allowing for ingest of data when certain storage is unavailable is described herein. The storage system includes zones that are independent and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When data is to be stored in the data storage system according to a specified storage policy and the specified storage policy cannot be achieved, the data is stored according to a fallback storage policy. This allows a client to be able to continue executing without having to wait for a storage anomaly to be corrected or pass. After the data is stored according to a fallback storage policy, the data is at a later time stored according to the specified storage policy.

Disconnected ingest in a distributed storage system

A data storage system having mutable objects incorporating time is described herein. According to the systems and methods described herein, a data item may be partitioned into parts (data objects) and stored as an index object. As the object storage system provides immutable objects, when a new version of a data item needs to be stored, only those parts (data objects) of the data item that changed need be saved rather than the entire data item. The systems and methods described herein allow for efficient storage, access and manipulation of mutable data items using an underlying immutable object system.

Data storage system having mutable objects incorporating time

Deduplication and compression evaluation methods and systems involve one or more processors obfuscating plain text file data in each file of a computer file system using a first cipher encryption scheme, obfuscating each plain text file name representing the plain text file data in each file of the computer file system using a second cipher encryption scheme, and associating each obfuscated file name representing the plain text file data of each of the plurality of files of the computer file system with the obfuscated file data of each of the plurality of files of the computer system. In addition, each plain text directory name for each of the obfuscated file names associated with the obfuscated file data in each of the plurality of files of the computer file system is obfuscated using a third cipher encryption scheme.

Data deduplication and compression evaluation methods and systems

Deduplication and compression evaluation methods and systems involve one or more processors generating a hash value for each block of data in a block storage device and creating a random substitution cypher using the respective hash values for each of said blocks of data as seed values to a random number generator. Thereafter, positions of run lengths for each of said blocks of data are randomly shuffled by the one or more processors with no run lengths having identical run length values disposed adjacent one another, a synthetic version of each of said blocks of data is generated using the substituted, shuffled, run lengths.

A failure resilient distributed replicated data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects calculated. Reassembly instructions are created for the data item. The data objects and parity objects are spread across all nodes and zones in the storage system. Reassembly instructions are also spread across the zones. When a read request is received, the data item is prepared from the lowest latency nodes according to the reassembly instructions. This provides for data resiliency while keeping the amount of storage space required relatively low.

Failure resilient distributed replicated data storage system

Asymmetric distributed replicated data storage systems and methods are described herein. The storage system includes zones that are independent, and autonomous. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects using erasure coding. The data objects and parity objects are spread across all nodes and zones in the storage system asymmetrically such that a first zone includes all of the data objects and no parity objects while the remaining zones include subsets of the data objects and all of the parity objects. The systems and methods provide for data resiliency while keeping the amount of storage space required relatively low.

Asymmetric distributed data storage system

Systems and methods for reducing metadata in a write-anywhere storage system are disclosed herein. The system includes a plurality of clients coupled with a plurality of storage nodes, each storage node having a plurality of primary storage devices coupled thereto. A memory management unit including cache memory is included in the client. The memory management unit serves as a cache for data produced by the clients before the data is stored in the primary storage. The cache includes an extent cache, an extent index, a commit cache and a commit index. The movement of data and metadata is by an interval tree. Methods for reducing data in the interval tree increase data storage and data retrieval performance of the system.

Reducing metadata in a write-anywhere storage system

A resilient distributed replicated data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects are calculated. Reassembly instructions are created for the data item. The data objects, parity objects and reassembly instructions are spread across nodes and zones in the storage system according to a policy for the data item. When a zone is inaccessible, a virtual zone is created and used until the intended zone is available. When a read request is received, the data item is prepared from the lowest latency nodes according to the reassembly instructions, and a virtual zone is accessed in place of a real zone when the real zone is inaccessible.

Resilient distributed replicated data storage system

A searchable data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, a local database is updated with information about the newly stored data item. When a search for a data item meeting certain metadata criteria is received, multiple concurrent searches are conducted across all storage devices in all nodes in all zones of the storage system. The configuration of the data storage system allows a parallel concurrent search at constituent storage devices to be performed quickly.

Searchable data in an object storage system

Data storage systems and methods for storing data are described herein. The storage system includes a first storage node is configured to issue a first delivery request to a first set of other storage nodes in the storage system, the first delivery request including a first at least one data operation for each of the first set of other storage nodes and issuing at least one other delivery request, while the first delivery request remains outstanding, the at least one other delivery request including a first commit request for each of the first set of other storage nodes. The first node causes the first at least one data operation to be made active within the storage system in response to receipt of a commit indicator along with a delivery acknowledgement regarding one of the at least one other delivery request.

Maintaining order and fault-tolerance in a distributed hash table system

Approaches for a digital storage device that moves or transforms data between various storage locations based on anticipated use. A digital storage device comprises one or more processors and one or more storage mediums for storing digital data. The digital storage device comprises a software agent. The agent maintains a local index to a set of data sets stored on the storage mediums. The indexed files are associated with an identifier, which may identity any unique entity. The software agent sends the local index over a network to an index manager. The agent receives, from the index manager, a remote index that identifies storage locations for other data sets associated with the identifier. The agent may use the local and remote index to move data sets between storage locations and/or transform data sets based on which device they will be accessed.

Growing community of inventors

Cupertino, CA, United States of America

Donald Joseph Molaro