Emc Corporation

Data Domain, Inc.

Other

Emc IP Holding Company LLC

Datrium, Linc.

R Hugo Patterson

Ming Benjamin Zhu

Kai Yi Li

Umesh Maheshwari

Edward Kihyen Lee

Sazzala Reddy

Allan J Bricker

Windsor Hsu

Richard Johnsson

Christopher R Lumb

Jeffery Zabarsky

Boris Weissman

Ganesh Venkitachalam

Brian Biles

Embodiments of a distributed virtual array data storage system and method are disclosed. Storage nodes made up of relatively unsophisticated disks with associated processors are scalable to store very large amounts of data. The storage nodes communicate with servers directly over a network through, for example, an Ethernet connection. Control of the storage nodes and access to the storage nodes is handled entirely on the server side of the system by distributed virtual array (DVA) software running on the server side and employing a particular protocol over the standard network connection. In an embodiment, server-side virtual machine (VM) hosts host application VMs that are associated with vDisks. The DVA software distributes data for the vDisk over the storage nodes. In the case of failure of one or more of the storage nodes, the DVA software reconstructs the data, for example by reading redundant data from surviving storage nodes.

Distributed virtual array data storage system and method

A method of managing duplicate segments from a segmented file storage system is disclosed. The method comprises indexing a segment according to a key for the segments wherein the index includes an identification of a first data location where the segment is stored and identifying a duplicate segment having the same key that is stored in a second location. The method further comprises determining that the duplicate segment is an undesired duplicate segment and eliminating the undesired duplicate segment.

Duplicate management

Techniques for efficiently indexing and searching similar data are described herein. According to one embodiment, in response to a query for one or more terms received from a client, a query index is accessed to retrieve a list of one or more super files. Each super file is associated with a group of similar files. Each super file includes terms and/or sequences of terms obtained from the associated group of similar files. Thereafter, the super files representing groups of similar files are presented to the client, where each of the super files includes at least one of the queried terms. Other methods and apparatuses are also described.

Efficiently indexing and searching similar data

Transmitting or storing subsegments is disclosed. A data stream or a data block is received and broken into a plurality of segments. For at least one segment, the segment is broken into a plurality of subsegments. A previously stored or transmitted segment similar to the at least one segment is identified. A fingerprint is computed for at least one subsegment. And, using the fingerprint for the at least one subsegment, determining whether the at least one subsegment is identical to a subsegment of the previously stored or transmitted segment without directly comparing the content of the at least one subsegment with the content of the subsegment of the previously stored or transmitted segment.

Subsegmenting for efficient storage, resemblance determination, and transmission

Techniques for providing access to data objects within another data object are described herein. In one embodiment, a compound object including multiple data objects is received and metadata is extracted for a data object from the compound object, where the metadata includes a layout of the data object in view of the compound object. Subsequently, access to one or more of the data objects within the compound object is provided based on the extracted metadata without using an application associated with the compound object. Other methods and apparatuses are also described.

Method and apparatus for providing access to data objects within another data object

A method of storing data is disclosed. A set of data blocks, including a plurality of proper subsets of data blocks, is stored. A plurality of first-level parity blocks is generated, wherein each first-level parity block is generated from a corresponding proper subset of data blocks within the plurality of proper subsets of data blocks without reference to other data blocks not in the corresponding proper subset. A second-level parity block is generated, wherein the second level parity block is generated from a plurality of data blocks included in at least two of the plurality of proper subsets of data blocks, and wherein recovery of a lost block in a given proper subset of data blocks is possible without reference to any data blocks not in the given proper subset.

Efficient redundant memory unit array

Storage of data segments is disclosed. For each segment, a similar segment to the segment is identified, wherein the similar segment is already managed by a cluster node. In the event the similar segment is identified, a reference to the similar segment and a delta between the similar segment and the segment are caused to be stored instead of the segment.

Cluster storage using delta compression

Selecting a segment boundary within block b is disclosed. A first anchor location j|j+1 is identified wherein a value of f(b[j&#x2212;A+1 . . . j+B]) satisfies a constraint and wherein A and B are non-negative integers. A segment boundary location k|k+1 is determined wherein k is greater than minimum distance from j.

Partitioning a data stream using embedded anchors

A method and apparatus for different embodiments of incremental garbage collection of data in a secondary storage. In one embodiment, a method comprises locating blocks of data in a log that are referenced and within a range at a tail of the log. The method also includes copying the blocks of data that are referenced and within the range to an unallocated segment of the log.

Incremental garbage collection of data in a secondary storage

Cluster storage is disclosed. A data stream or a data block is received. The data stream or the data block is broken into segments. For each segment, a cluster node is selected, and a portion of the segment smaller than the segment is identified that is a duplicate of a portion of a segment already managed by the cluster node.

Growing community of inventors

Mountain View, CA, United States of America

R Hugo Patterson