Emc IP Holding Company LLC

Dell Products Inc.

Mayank Rajawat

Barry Naujok

Chris Stacey

Eric Michael Lemar

Anton Rang

Max Laier

Benjamin A Leis

Matthew D Fleming

Harsha Mahuli

Mohd Fahadullah

Jonathan Walton

Sourav Ghosh

Jeffrey Tremaine

Dhanashri Parasharam Patil

Kiran Ballal

Simarjot Kaur

The described technology is generally directed towards maintaining a heat score per file based on a file's access/activity level, and using the heat score to re-tier files from faster to slower storage and vice-versa. Files on a slower tier that have more activity are thereby moved ('up-tiered') to a faster access storage tier, while the files that have less activity on the faster access storage tier are moved (“down-tiered”) to a slower access storage tier. The heat score for a file is increased based on more input-output operation activity, and decayed over time. The decaying can be based on a relative importance value (e.g., business value) assigned to the file or a domain that includes the file. Ordering the files based on their respective heat scores allows only a fraction of the files to be considered for re-tiering before the re-tiering moves are performed.

Heat score-based tiering of data between different storage tiers of a file system

The described technology is generally directed an extension to the IFS domains architecture, referred to herein as filter domains. IFS domains allows tagging of files in a tree-like dataset. Thus, a domain can be defined at the root of the dataset such as the topmost directory under which all files reside. These domains are inherently hierarchical, path-based entities. Filter domains extends this organization to allow domains to be applied beyond hierarchical tree structures in order to also provide arbitrary grouping of file objects based on any suitable membership condition.

Extending filesystem domains with a domain membership condition

The described technology is generally directed towards domains that data services can use to collect files of a global filesystem namespace into groups, and domains that can be used to exclude certain files or metadata. A data service (a domain patron) creates a domain, e.g., for a particular directory, and a domain manager associates files under that directory with domain membership information. The data service can relate exclusion information to the domain, e.g., to exclude part of the directory. This can be done by an exclusion domain instance, e.g., a child to a parent domain, or can be done in metadata, by having the data service (e.g., a snapshot data service) exclude certain metadata such as snapshot identifiers from a larger set of snapshot identifiers.

Excluding directory trees from data service operations

The described technology is generally directed towards domains that data services can use to collect files of a global filesystem namespace into groups. A data service (a domain patron) creates a domain, e.g., for a particular directory, and a domain manager associates files under that directory with domain membership information. Thereafter, the data service can use the domain membership information associated with a file to determine whether to include that file in a data service operation. In one implementation the membership information is maintained in file metadata, facilitating fast and efficient retrieval of the information in near constant time. Exclusion from a domain, hard links, renames and alternate data streams are among the various aspects supported by the domain technology.

Partitioning a distributed filesystem namespace

Implementations are provided herein for data deduplication, and more particularly, to post-process data deduplication on a large scale out storage system. Multiple techniques and implementations are disclosed that offer greater efficiency, higher performance, and more stability when performing post-process data deduplication at large scale. Disclosed implementations are based on a process for data deduplication involving four main phases: enumeration, commonality, sharing, and update. Multi-level hashing can be used to identify candidates for deduplication during the enumeration phase, providing a more efficient use of compute resources. In addition, datasets can be phase rotated through the post-process deduplication steps providing a more controllable deduplication environment as well as a more efficient use of resources.

Scalable post-process deduplication

Growing community of inventors

Issaquah, WA, United States of America

Mayank Rajawat