Quantum Corporation

Other

Roderick B Wideman

Turguy Goker

Don Doerner

Suayb Sefik Arslan

Jaewook Lee

Michael Zeis

Stephen W Moore

Jeff Leuschner

Doug Burling

Barry Massey

John Rockenfeller

Stephen P Lord

Greg Wade

Carsten H Prigge

Matthew C Dewey

Camden John Davis

John Theodore Kline

Steve Lord

Douglas Alan Burling

Joshua Fitzsimmons Martin

Don Doemer

Chris Cason

Darrel Somer

Darryl A Torske

Brian Sunnen

Robert A Yang

Jeffrey Szmyd

Jeffrey D Leuschner

A method for implementing an object store using removable storage media includes the steps of receiving a request to retrieve first data; determining a first data object in which at least a portion of the first data is stored; determining a removable storage medium on which the first data object is stored, the removable storage medium including a value store partition into which one or more data objects including at least the first data object are stored and a key store partition into which one or more keys identifying the one or more data objects are stored; retrieving the first data object from the value store partition; and retrieving the at least a portion of the first data from the first data object. The method can further be performed using a data management system and/or one or more non-transitory computer-readable storage media.

Removable media based object store

Embodiments disclosed herein provide systems, methods, and computer-readable media to implement an object store with removable storage media. In a particular embodiment, a method provides identifying first data for storage on a first removable storage medium and designating at least a portion of the first data to a first data object. The method further provides determining a first location where to store the first data object in a first value store partition of the first removable storage medium and writing the first data object to the first location. Also, the method provides writing a first key that identifies the first data object and indicates the first location to a first key store partition of the first removable storage medium.

Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Filter file system with inode number as primary database key

Methods and apparatus deduplicate and erasure code a message in a data storage system. One example apparatus includes a first chunking circuit that generates a set of data chunks from a message, an outer precoding circuit that generates a set of precoded data chunks and a set of parity symbols from the set of data chunks, a second chunking circuit that generates a set of chunked parity symbols from the set of parity symbols, a deduplication circuit that generates a set of deduplicated data chunks by deduplicating the set of precoded chunks or the set of chunked parity symbols, an unequal error protection (UEP) circuit that generates an encoded message from the set of deduplicated data chunks, and a storage circuit that controls the data storage system to store the set of deduplicated data chunks, the set of parity symbols, or the encoded message.

Joint de-duplication-erasure coded distributed storage

Embodiments include a data aware deduplicating object store. The data aware deduplicating data store includes a consistent hashing logic that manages a consistent hashing architecture for the object store. The consistent hashing architecture includes a metadata ring and a bulk ring. The consistent hashing architecture may be a multiple ring architecture comprising a metadata ring and two or more bulk rings. A bulk ring may include a key/value (k/v) data store, where a k/v data store stores a shard of an index and a reference count that facilitates the individual approach to garbage collection or data reclamation. The data aware deduplicating data store also includes a deduplication logic that provides data deduplication for data to be stored in the object store. The deduplication logic performs variable length deduplication and provides a shared nothing approach.

Data aware deduplication object storage (DADOS)

An example method includes determining a configuration of two or more partitions for a sequential access medium. At least one partition stores data de-duplication data structures while at least one other partition stores a repository of unique data blocks associated with the data structures. The method also includes controlling a data de-duplication computer to configure the sequential access medium according to the configuration. The method includes producing an output sequence for writing the data structures and a set of unique data blocks associated with the set of data structures to the sequential access medium as configured with the two or more partitions. One embodiment includes controlling a data de-duplication computer to write the data de-duplication data structures and the set of unique data blocks to the sequential access medium according to the output sequence.

Sequential access storage and data de-duplication

Example apparatus and methods combine erasure coding with data deduplication to simultaneously reduce the overall redundancy in data while increasing the redundancy of unique data. In one embodiment, an efficient representation of a data set is produced by deduplication. The efficient representation reduces duplicate data in the data set. Redundancy is then added back into the data set using erasure coding. The redundancy that is added back in adds protection to the unique data associated with the efficient representation. How much redundancy is added back in and what type of redundancy is added back in may be controlled based on an attribute (e.g., value, reference count, symbol size, number of symbols) of the unique data. Decisions concerning how much and what type of redundancy to add back in may be adapted over time based, for example, on observations of the efficiency of the overall system.

Data deduplication with adaptive erasure code redundancy

Example apparatus and methods provide improved reclamation, garbage collection (GC) and defragmentation (defrag) for data storage devices including solid state drives (SSD) or shingled magnetic recording (SMR) drives. An erasure code (EC) layer that facilitates logically or physically erasing data from the SSD or SMR as a comprehensive GC or defrag is added to the SSD or SMR. Erased data may be selectively recreated from the EC layer as needed. Pre-planned EC write zones may be established to further optimize GC and defrag. Recreated data may be written to selected locations to further optimize SSD and SMR performance. Erasure code data may be distributed to co-operating devices to further improve GC or defrag. Example apparatus and methods may also facilitate writing data to an SMR drive using tape or VTL applications or processes and providing a pseudo virtual tape library on the SMR drive.

Garbage collection and defragmentation for solid state drives (SSD) and shingled magnetic recording (SMR) drives

Example apparatus and methods provide a file system that stores files in multiple different logical units. The logical units are associated with different classes of devices. The file system interacts with the logical units through block-based interfaces. The file system is a single file system that spans the multiple logical units. The multiple different logical units include data storage devices for which the file system manages block storage allocation and an object storage system that manages its own block storage allocation rather than having the file system manage its block storage allocation. The file system may maintain a single namespace, hierarchy, or directory structure for the multiple logical units. The object storage system may protect files using redundant copies, erasure codes, or other approaches. An archive process may selectively move files from other devices to the object storage system and leave a symbolic link behind.

File system with multi-class in situ tiered archiving

Example apparatus and methods access a Linear Tape File System (LTFS) repository that stores data associated with an external entity (e.g., file system, application). Data is copied from the LTFS repository, selectively converted to a form associated with a cloud repository, and provided to the cloud repository. A mapping between corresponding addresses in the LTFS repository and cloud repository is established. Requests by the external entity for data may be satisfied from either the LTFS repository or the cloud repository until a changeover event is detected and are then satisfied from the cloud repository afterwards. The external entity runs on a different computer than the method. The method is transparent to the external entity, is performed independent of the external entity, and continues under its own control until a selected amount of data is moved from the LTFS repository to the cloud repository.

Moving data from linear tape file system storage to cloud storage

Example methods and apparatus associated with a messaging policy controlled email deduplication are provided. In one example a messaging policy is accessed. It is determined whether a received message complies with the policy based on rules of the messaging policy. If a message complies with the messaging policy, the message is displayed. If the message does not comply with the messaging policy, it is determined whether the message is duplicative. If the message is deemed duplicative it is not displayed. Conversely, if the message is not deemed duplicative it is displayed.

Growing community of inventors

Shakopee, MN, United States of America

Roderick B Wideman