Rubrik, Inc.

Emc IP Holding Company LLC

Abdullah Reza

Vijay Karthik

Guilherme Menezes

Bristy Sikder

Siddharth Bidasaria

Fabiano C Botelho

Anshul Mohan Gupta

Guilherme Vale Ferreira Menezes

Abhinav Duggal

Lan Bai

Teng Xu

Ishaan Sang

Nitin Rathor

Vaibhav Gosain

Arjun Sinha

Methods, systems, and devices for data management are described. A data management system (DMS) may access a snapshot of a target client object to perform a block-order traversal of a set of files included in the snapshot. The DMS may generate an index file for the set of files based on a metadata associated with the snapshot, the index file including a mapping between a file path for each file and one or more properties of each file. The DMS may then read the set of files included in the snapshot using a sliding window in accordance with the mapping between the file path for each file and the one or more properties of each file, and may determine whether to generate an alert based on reading the set of files included in the snapshot.

Techniques for block-order traversal of files

A storage system may store one or more snapshots of a computing system to support backup and restoration of data stored at the computing system. The storage system may identify an expiration of a first snapshot indicating a first set of physical storage locations to which first data of the computing system was stored as part of a first backup procedure. The storage system may identify a first subset physical storage locations of the first set as storing a first portion of the first data that is superseded by second data associated with a second snapshot. Based on identifying the first subset, the storage system may delete the first portion of the first data from the first subset of physical storage locations and retain a second portion of the first data at a second subset of physical storage locations.

Backup data consolidation

A method for recovering files from a filesystem stored across sparse files in a cloud environment is described. According to the method, a data management system may receive a request to read the files. The data management system may identify one or more target address ranges corresponding to the files indicated via the request. The data management system may read index information for the sparse files in the cloud environment. The index information may indicate respective address ranges for data blocks within the sparse files. The data management system may identify one or more data blocks within one or more sparse files as corresponding to address ranges that overlap with the one or more target address ranges based on the index information. The data management system may transmit, to the cloud environment, one or more read requests for the identified one or more data blocks.

Multi-phase file recovery from cloud environments

A method of consolidating snapshots includes receiving a request to consolidate a first snapshot with a second snapshot into a third snapshot, the first and second snapshots stored in separate backup files, each backup file organized as a directory where data parts of the first and second snapshots can be hard linked to locations outside of the backup file, comparing the data parts of the first and second snapshots to determine if any second snapshot data part fully overlaps with any first snapshot data part, responsive to determining that a second snapshot data part fully overlaps with a first snapshot data part, hard linking the determined second snapshot data part into the third snapshot, and storing the third snapshot in the backup file.

Consolidating snapshots using partitioned patch files

In some examples, a method comprises: receiving a request to read data within a specified range from a backup file storing at least one base snapshot and at least one incremental snapshot; looking up the specified range in range filters from the backup file, the range filters corresponding to snapshots stored in the backup file and each range filter comprising bits indicating whether data exists at respective ranges within the snapshot corresponding to the respective range filter; and in response to the looking up, reading the requested data from the looked-up range in the backup file.

Snapshot range filters

In some examples, a data management and storage (DMS) platform, comprises peer DMS nodes in a node cluster, a distributed data store comprising local and cloud storage, and at least one processor configured to perform operations in a method of creating a local consolidated patch file from a patch file chain stored in the cloud storage. The operations include, in a first dry-run phase, creating a logical patch file image of data blocks in one or more cloud patch files stored in the cloud storage; in a second data-transfer phase, downloading at least some of the data blocks from the cloud patch files identified by the logical patch file image, the second data-transfer phase comprising a coalescing operation to construct a set of coalesced reads of the data blocks; and creating and storing, in the local storage, the local consolidated patch file using the downloaded data blocks.

Smart coalescing in data management systems

In some examples, a data management and storage (DMS) platform comprises peer DMS nodes in a node cluster, a distributed data store comprising local and cloud storage, and at least one processor configured to perform operations in a method of creating a local consolidated patch file from a patch file chain stored in the cloud storage. Example operations comprise, in a first dry-run phase, creating a patch file image of data blocks in one or more cloud patch files stored in the cloud storage; in a second data-transfer phase, downloading at least some of the data blocks from the cloud patch files identified by the patch file image; and creating and storing, in the local storage, the local consolidated patch file using the downloaded data blocks.

Two-phase snapshot recovery

In some examples, a data management system generates snapshots in a distributed file system based on a protocol or a user triggered event, The data management system identifies a snappable file in a distributed file system and a first data block in the snappable file, the first data block including data and attribute data. The system scans an index file to access the attribute data of the first data block and initiates construction of a patch file based on the accessed attribute data. The system repeats the scanning of the index file to access attribute data of at least a further second data block, the second data block including data and attribute data, and completes construction of the patch file based on the accessed attribute data of the first and second data blocks. The system generates conversion simulation information by collecting attribute data for all the data blocks of the constructed patch file, and writes the simulation information to a patch file image.

Online data format conversion

A lightweight deduplication system can perform resource efficient data deduplication using an extent index and a content index. The extent index can store full fingerprints of data segments to be deduplicated and the content index can store shortened versions of the full fingerprints. The system can alternate between the extent and content indexes, and cache portions of the indices to perform lightweight data deduplication. Further, the system can be configured with an efficient heuristic approach for selecting content index data lookups for chains of volumes for deduplication, such as a long chain of snapshots.

Light-weight index deduplication and hierarchical snapshot replication

In one example, a method includes measuring an amount of physical storage space used, or expected to be used, by a portion of a dataset S of segments, and measuring the amount of physical storage space includes receiving information that identifies an ad-hoc group of size 'n' of files F. . . Fthat makes up a subset of the dataset S, determining a number of unique segments in the dataset S, identifying a respective unique segment set U. . . Ufor each of the 'n' files in the ad-hoc group of files, performing a set union operation on the unique segment sets U. . . U, and determining a sum of sizes of the unique segment sets U. . . U, where the sum is the amount of physical storage space used or expected to be used by the ad-hoc group of size &#x2018;n&#x2019; of files F. . . F.

System and method for efficiently measuring physical space for an ad-hoc subset of files in protection storage filesystem with stream segmentation and data deduplication

Growing community of inventors

Santa Clara, CA, United States of America

Abdullah Reza