Emc IP Holding Company LLC

Symantec Corporation

Murthy V Mamidi

George Mathew

Abhishek Rajimwale

Charles J Hickey

Pengju Shang

Neerajkumar Nareshkumar Chourasia

Donna Barry Lewis

Pranay Singh

Kadir Ozdemir

Charles Silvers

Paul Massiglia

Ronald S Karr

Anindya Banerjee

Raghupathi Malige

Xiaohong W Wang

Englin Koay

Gautham Ravi

Sai Chivukula

Sirisha Kapoor

Geetha Shendge

Charles I Hickey

Sampath Jayaram

Described is a system (and method) that maintains deduplication efficiency when storing data within a clustered data storage environment that implements a global namespace. To provide such a capability, the system may obtain granular data source identifying information from a client system that provides data to be backed-up by a backup component. The data source identifying information may take the form of a placement tag that is associated with the received data. The backup component may then provide such placement tags when providing the backup data to the clustered storage system. The placement tags may then be used to intelligently distribute backup files to particular storage nodes of the clustered storage system to improve deduplication efficiency.

Granular data source identification for obtaining deduplication storage efficiency within a clustered environment

A method of generating a hybrid snapshot includes receiving a request to generate a snapshot of a distributed file system and identifying a first storage resource of the distributed file system and a second storage resource of the distributed file system based on the request. The method further includes generating the snapshot of the distributed file system, the snapshot including a data-full snapshot of the first storage resource and a data-less snapshot of the second storage resource.

Hybrid snapshot of a global namespace

Embodiments for enabling granular migration of data with high efficiency. A defined metadata element, a tag, is assigned to each file, and then tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer, and such a group of tags is referred to as a tag set. Embodiments can be used with a defined backup system file migration process, such as present in the Data Domain File System. By using snapshots, incoming new data (ingested file) is allowed to continue while the migration is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method is efficient became it makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Granular data migration

Embodiments for granular replication of data with high efficiency. A defined metadata element embodied as a tag is assigned to each file. Tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer. Embodiments can be used with a defined backup system file replication process, such as present in the Data Domain File System. By using snapshots, incoming new data is continued to be ingested while the replication is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method efficiently makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Granular data replication

A method of capturing a fractional snapshot of a distributed file system includes receiving a snapshot request to generate a factional snapshot of a file system. The request includes an indication of a subset of data of the file system. The method further includes identifying, by a processing device, the subset of data of the file system indicated by the snapshot request and generating the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.

Fractional consistent global snapshots of a distributed namespace

Described is a system (and method) that intelligently distributes data within a clustered storage environment. To provide such a capability, the system may distribute backup files by considering a source of the data to be backed-up. In particular, the system may leverage the ability of front-end components such as a backup application to perform a granular data source identification of data. Such information may be propagated to back-end components such as a storage filesystem in the form of a data source identifier (e.g. placement tag). The data source identifiers may then be accessed by the clustered storage system to intelligently distribute backup files amongst a set of storage nodes forming a cluster. For example, backup files from the same data source may be stored on the same storage node to obtain the same deduplication efficiency as a single storage system.

Utilizing data source identifiers to obtain deduplication efficiency within a clustered storage environment

Embodiments for enabling snapshot backups in a global name space of a cluster network, by representing the name space of cluster network in an MTree, storing data files organized in a B+ Tree format on one or more data nodes, storing name specific information of the data files in a B+ Tree format in a meta node, wherein a B+ Tree of the meta node accesses each corresponding B+ Tree in each of the one or more data nodes. The process takes snapshot backups of individual MTree limbs, and links the limbs of each snapshot into groups based on a cluster identifier and snapshot identifier.

Global snapshot backups of a distributed name space

Disclosed herein are techniques for generating fractional checkpoints on a distributed file system by snapshotting subcomponents referred to as 'file sets.' The techniques include capturing a present time; calculating from configured parameters a future wakeup time for a subsequent set of checkpoints from the present time; walking a database of meta file set objects to locate at least one meta file set object; calculating a retention period for a snapshot that is being created; and generating a global data-less snapshot for the meta file set object and remote data file set objects associated with the meta file set object, and then repeating the process for subsequent file set objects in the database.

System and method of generating automatic checkpoints of a distributed file system

Embodiments for providing global inline name space verification for a distributed file system in a network of a metadata server coupled to a plurality of data servers by taking a global dataless snapshot of a namespace of the distributed file system; walking all of the files in the namespace for each data server and the metadata server to generate parsed information; combining, by an XOR operation, the parsed information into data blocks for each server; obtaining a checksum of each data block of the data blocks; comparing actual and expected checksums from the metadata server and all of the data servers; and generating an alert if a comparison of any actual and expected checksums do not match.

Global inline name space verification for distributed file systems

Described is a system that provides the ability to distribute backup data according to performance characteristics associated with nodes that form a clustered storage environment. For example, the clustered storage environment may organize nodes into tiers that have certain storage performance-related characteristics such as read/write latency, deduplication processing efficiency, bandwidth, etc. Accordingly, the system may associate the backup data with an identifier for an appropriate storage performance tier. The identifier may then be utilized by the clustered storage environment to redirect storage operations to a node within the appropriate storage performance tier. Thus, the system allows for the distribution of backup data to particular storage performance tiers based on particular storage or workload requirements.

Storage performance-based distribution of deduplicated data to nodes within a clustered storage environment

Embodiments for optimizing file lookups after a file or directory rename operation. Files are defined using inode, dirent, and hash entries, and the name space of the files are stored in a B&#x2212;Tree using file system conventions in which the inode, dirent, and hash values are referenced by the directory first followed by the inode number, as in a first data structure denoted [directory:inode]. Upon a rename of a file, the old inode, dirent, and hash entries are deleted, and a reference is created to the old name using the inode number first followed by the old directory, as in a second data structure denoted [inode:old_directory], and this reference is used as a forward link to the new file under the new directory name.

Efficient file renames using B-tree based persistence for file system name spaces

In one example, a method includes receiving metadata in the form of a modification to metadata represented by a file system namespace abstraction, wherein the file system namespace abstraction corresponds to less than an entire file system namespace, and the file system namespace abstraction includes one or more pages, and one of the pages corresponds to a particular cached block, updating the file system namespace abstraction based on the received metadata, determining if caching is enabled for the file system namespace abstraction, when caching is enabled for the file system namespace abstraction, caching the updated file system namespace abstraction in SSD storage that includes the cached block, and maintaining a status of the cached block in a Dtable of the SSD storage.

Namespace performance acceleration by selective SSD caching

Described is a system for reclaiming unused storage space when distributing backup data to storage performance-based tiers within a clustered storage environment. The clustered storage environment may organize nodes into storage performance tiers, and each node within a tier may be tuned for a particular type of workload. Accordingly, the system implements a file system that provides the ability to reclaim unused storage space (e.g. perform garbage collection) for specific nodes or tiers independently despite implementing a global namespace. The global namespace may group (or co-locate) aspects of file information with the backup files. For example, file information may be stored on the same node that stores the associated backup files. Accordingly, the system may reclaim unused storage space for each node, or set of nodes forming a storage performance tier, independently and efficiently.

Efficient name space organization in a global name space cluster

The distribution of global namespace to achieve performance and capacity linear scaling in cluster filesystems. Specifically, the disclosure entails the distribution of files, tracked in namespace trees, among various cluster nodes while linking the namespace trees through a centralized access point, thus providing a global namespace abstraction for the distributed cluster.

Distribution of global namespace to achieve performance and capacity linear scaling in cluster filesystems

In one example, a method includes receiving metadata in the form of a modification to metadata represented by a file system namespace abstraction, and the file system namespace abstraction corresponds to less than the entire file system namespace. Next, the file system namespace abstraction is updated based on the received metadata. Next, a determination is made whether or not caching is enabled for the file system namespace abstraction. If caching is enabled for the file system namespace abstraction, the updated file system namespace abstraction is cached in SSD storage.

Growing community of inventors

San Jose, CA, United States of America

Murthy V Mamidi