Sepaton, Inc.

Hitachi Vantara Corporation

Hitachi Data Systems Corporation

Ronald Ray Trimble

Jon Christopher Kennedy

Carey Jay McMaster

Timmie G Reiter

Stefan Merrill King

David Biernacki

Jeffrey Vincent Tofano

Patrick James Grinwald

Thomas R Ramsdell

John Henry Petrangelo

Roland Leo Sorel

In some examples, a computer system may receive a plurality of chunks of data of a data object. The system may compress the plurality of chunks of data to obtain a plurality of compressed chunks, and may determine whether the plurality of compressed chunks together are less than a threshold size. Based on determining that the plurality of compressed chunks together are less than the threshold size, the system may add, to respective entries in a map data structure, respective sizes of the plurality of compressed chunks. In addition, the system may compact the map data structure by combining values in at least two of the respective entries, and may store the plurality of compressed chunks and the compacted map data structure.

Optimizing storage and retrieval of compressed data

In some examples, a computing device may be configured to store data to storage subsystems in a storage pool. The computing device may determine, for each storage subsystem, a performance potential quantity representative of a relative performance of the storage subsystem, and may determine a respective initial distribution weighting for each storage subsystem based at least on the performance potential quantities. Further, the computing device may determine, for each storage subsystem having a percentage of storage capacity used that is more than a threshold amount greater than a percentage of storage capacity used by the storage subsystem having a lowest percentage of storage capacity used, one or more amounts by which to adjust the initial distribution weighting to determine a respective desired distribution weighting for each storage subsystem. The computing device may select one or more storage subsystems to receive data based at least on the desired distribution weightings.

Capacity and performance optimization in non-homogeneous storage

A system configured to compute match potential between first data and second data is provided. The system includes data storage storing the first data and the second data, and at least one processor coupled to the data storage. The at least one processor is configured to identify a first sequence of fingerprints characterizing a first plurality of sections of the first data, the first sequence being ordered according to an order of the first plurality of sections within the first data; identify a second sequence of fingerprints comprising fingerprints that match fingerprints within the first sequence, the second sequence of fingerprints characterizing a second plurality of sections of the second data, the second sequence being ordered according to an order of the second plurality of sections within the second data; quantify a similarity between the first sequence and the second sequence; and adjust the match potential based on the similarity.

Systems and methods of locating redundant data using patterns of matching fingerprints

According to at least one embodiment, a data storage system is provided. The data storage system includes memory, at least one processor in data communication with the memory, and a deduplication director component executable by the at least one processor. The deduplication director component is configured to receive data for storage on the data storage system, analyze the data to determine whether the data is suitable for at least one of summary-based deduplication, content-based deduplication, and no deduplication, and store, in a common object store, at least one of the data and a reference to duplicate data stored in the common object store.

Multi-node hybrid deduplication

A system configured to generate a macro-fingerprint from at least one predefined set of summaries is provided. The system includes data storage storing a first predefined set of summaries associated with a first region of data, each member of the first predefined set of summaries characterizing data within the first region of data; and at least one processor coupled to the data storage and configured to: read the first predefined set of summaries; select at least one first member from the first predefined set of summaries based on a value of the at least one first member; and store the at least one first member within a first macro-fingerprint. The first region of data may have a first size indicative of a quantity of data included in the first region of data. The macro fingerprints are created from previously created smaller (micro) fingerprints without having to reread the data.

System and method for multi-scale navigation of data

Described are computer-based methods and apparatuses, including computer program products, for detection and deduplication of backup sets exhibiting poor locality. A first set of summaries of a first data set are determined, each summary of the first set of summaries being indicative of a data pattern in the first data set. A second set of summaries of a second data set are determined, each summary of the second set of summaries being indicative of a data pattern in the second data set. A set of comparison metrics are calculated, each comparison metric being based on a first subset of summaries from the first set of summaries and a second subset of summaries from the second set of summaries. A locality metric is calculated based on the set of comparison metrics indicative of whether the first data set and second data set exhibit poor locality.

Detection and deduplication of backup sets exhibiting poor locality

Described are computer-based methods and apparatuses, including computer program products, for dynamic deduplication. Data is processed using an algorithm that deduplicates the data based on a first set of parameters. A first moving average of a first deduplication performance metric is calculated for the algorithm over a time period. A second moving average of a second deduplication performance metric is calculated for the algorithm over the time period. It is determined that the first moving average satisfies a first criterion, the second moving average satisfies a second criterion, or both. The algorithm is reconfigured based on a second set of parameters to deduplicate data.

Dynamic deduplication

Described are computer-based methods and apparatuses, including computer program products, for removing redundant data from a storage system. In one example, a data delineation process delineates data targeted for de-duplication into regions using a plurality of markers. The de-duplication system determines which of these regions should be subject to further de-duplication processing by comparing metadata representing the regions to metadata representing regions of a reference data set. The de-duplication system identifies an area of data that incorporates the regions that should be subject to further de-duplication processing and de-duplicates this area with reference to a corresponding area within the reference data set.

Growing community of inventors

Acton, MA, United States of America

Ronald Ray Trimble