Emc Corporation

Emc IP Holding Company LLC

Dell Products Inc.

Morgan A Clark

Jean-Pierre Bono

Michael Scheer

Sairam Veeraswamy

Ahsan Rashid

William C Davenport

Rohit K Chawla

Marc A De Souter

Alexander S Mathews

Marc DeSouter

Pranit Sethi

Philippe Armangau

Walter Forrester

Michael D Schouten

Uresh K Vahalia

John Michael Hayden

Soumyadeep Sen

Sachin Mullick

Miles Aram De Forest

Peter C Bixby

Ivan Basov

Yingchao Zhou

Christopher H Stacey

Srinivasa Rao Vempati

Hongliang Tang

Qi Mao

Yunfei Chen

Dixit Patel

Saurabh Ramchandra Godbole

Jean Pierre Bono

Maher Kachmar

A method facilitating an update to a file system comprises storing a set of transaction log entries at log space of a storage log associated with the file system, the transaction log entries comprising instructions for fulfilling a set of transactions that define change to the file system, assigning, in an ordered sequence corresponding to a sequence order of journalling of the set of transaction log entries to the log space, transaction markers to a set of transaction marker lists, wherein the transaction markers reference the set of transaction log entries, and, for individual transaction markers of the transaction markers, preventing reuse of the log space corresponding thereto until an evaluation of the set of transaction marker lists results in a determination that writing of all modified system metadata corresponding thereto has been executed and that the transaction marker is set equal to a global head transitioning the transaction markers.

Multiple bitmap single head/tail retirement transaction order

Described is technology that facilitates journalling a transaction log entry to chunks of storage while allowing for obtaining of modified system metadata associated therewith in the case of a computing device issue. An associated method comprises at least storing, by a system operatively coupled to a processor, a set of transaction log entries at log space of a storage log associated with a file system, wherein the transaction log entries comprise instructions for fulfilling a set of transactions, and wherein the transactions define changes to the file system, generating a checkpoint marker corresponding to a set of log storage objects of the storage log at which the transaction log entries are stored, and based on the checkpoint marker, preventing, by the system, reuse of the log space until writing of all modified system metadata corresponding to the transaction log entries has been executed.

Multi-log checkpoint-based retirement transaction order in multi-chunk log

The technology described herein efficiently determines whether a real inode is shared among views, or owned. In-memory data structures include a view snapshot generation counter that is increased as a snapshot that generates a view is created, and an inode total weight. An in-memory virtual inode cache dataset for a filesystem object associated with the view is instantiated with the value of snapshot generation counter, sharing-related data based on the inode mapping file entry for the object, and an inode access weight. To determine whether the inode is shared (and needs to be split), such as on a write to the object, the in-memory data is evaluated. The real inode is shared if the generation counters are unequal, if the sharing-related data indicates sharing at an intermediate indirect block level, or indicates sharing at the inode level and the inode access weight is less than the inode total weight.

Determining a sharing relationship of a file in namespace snapshots

Technology described herein can perform deletion of a snapshot or portion thereof. In an embodiment, a system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, to delete a snapshot, or a portion of a snapshot, of a real filesystem, reading an inode mapping file (IMF) of the snapshot that indexes a virtual inode number (VIN) corresponding to a real inode. The operations further can comprise identifying the real inode of the snapshot referenced by the VIN, identifying a file object corresponding to the real inode, and deleting the file object from the snapshot.

Deletion of snapshot of filesystem

Optimizing file system resource reservation is presented herein. The method comprises dividing a virtual file system address space into subspaces, initializing the subspaces with volume slices of a group of volume slices comprising a first volume slice, a second volume slice, and a collection of reserved volume slices allocated based on an allocation pattern that allocates volume slices as a function of a quantitative relationship between a first value associated with a first volume slice and a second value associated with a second volume slice, determining that a data block count is insufficient to service a write operation of user data to the second volume slice; and provisioning a second subspace with a free volume slice obtained from the collection of reserved volume slices, and wherein the provisioning of the second subspace with the free volume slice is performed without invoking a memory exclusion mechanism.

Optimizing file-system resource reservation

Techniques for limiting storage consumed by a file system without shrinking a volume upon which the file system is deployed. The techniques are employed in a clustered environment including multiple NAS nodes, each having access to block storage including multiple storage devices. By deploying the file system on a volume of a NAS node within the clustered environment, setting the value of the FS user size to be equal to the FS volume size, and if, at a later time, it is desired to reduce the file system size, setting the value of the FS user size to a lesser value than the FS volume size, IO requests received at the NAS node can be satisfied within the logical limit of the lesser value of the FS user size without shrinking the local volume, allowing the file system size to be reduced without requiring close coordination with the block storage.

Techniques for limiting the maximum storage consumed by a file system without shrinking an underlying volume

Described are methods, systems, and apparatus, including computer program products for de-duplicating access control lists (ACLs). A first ACL associated with a first computer file is received. A first checksum based at least in part on the first ACL is calculated. One or more directory entries based on the first checksum are retrieved from a de-duplication directory, wherein each directory entry of the one or more directory entries comprises a reference to an ACL and a name comprising the first checksum. A directory entry of the one or more directory entries is identified that references a second ACL that specifies the same permissions as the first ACL. A reference to the second ACL of the directory entry is added to the first computer file.

De-duplication of access control lists

An improved technique involves identifying the location of backed free blocks, i.e., blocks within an upper deck file system that have been provisioned from storage devices of a data storage system to a lower deck file system, allocated from the lower deck file system to the upper deck file system, and later freed from the upper deck file system. A storage processor accesses a set of data structures that identifies backed free blocks as opposed to free blocks that have not been written into and thus do not correspond to any allocated blocks in storage. Once the storage processor identifies the backed free blocks, the storage processor frees each block in the lower deck file system from which the respective backed free block was allocated. The storage processor then updates the set of data structures to indicate that the respective backed free block is now simply a free block.

Efficient scavenging of data and metadata file system blocks

Directory attributes are provided so that the time for creating an incremental backup of a file system by a scan of the file system tree is proportional generally to the number of files that change between backups instead of the number of files in the file system. A tree modification attribute indicates whether or not any file in a directory tree has changed since the last backup. If no file has changed in the directory tree, then the entire tree is skipped during the scan for changed files. In addition, a list is compiled of the files in the directory that represent branches having at least one changed file, so that the list is scanned instead of the directory entries. When a file is changed for the first time since the last backup, the file is queued so that the directory attributes are updated in a background process.

Rapid incremental backup of changed files in a file system

A technique for managing storage tiering in a data storage apparatus operates at a fine level of granularity in response to write requests as blocks are allocated for writing. As write requests arrive or are otherwise processed by the data storage apparatus, the data storage apparatus identifies a quality of service to be provided for satisfying each write request and allocates a set of blocks from storage devices of storage tiers that provide the identified quality of service. The data storage apparatus may then store the information specified in the write request in the newly allocated blocks.

File system inline fine grained tiering

A technique for managing a file system includes dividing the addressable space of a file system into multiple subspaces: a data subspace for user data; an inode subspace for inodes describing the user data; and an MDB (metadata block) subspace for indirect blocks and/or directory structures pertaining to the user data. In response to storage requests to write user files to the file system, the file system stores user data of the files in the data subspace, stores inodes pertaining to the user files in the inode subspace, and stores indirect blocks and/or directories in the MDB subspace.

Segregating data and metadata in a file system

A method, computer program product, and computing system for grouping storage blocks within a file system into a plurality of storage pools including a free-backed storage pool, a free-unbacked storage pool, and an allocated-backed storage pool. The free-backed storage pool identifies unused storage blocks within the file system that are already associated with physical storage space within a backend storage system. The free-unbacked storage pool identifies unused storage blocks within the file system that are not yet associated with physical storage space within the backend storage system. The allocated-backed storage pool identifies used storage blocks within the file system that are already associated with physical storage space within the backend storage system. A request is received for one or more unused storage blocks within the file system. The request is fulfilled by selecting one or more unused storage blocks identified within the storage pools while prioritizing the selection of unused storage blocks identified within the free-backed storage pool.

System and method for thin provisioning

An improved technique generates formatted file system metadata in increments only when metadata for those increments are specifically requested. A location on a storage volume designated for storing file system metadata is read and tested to determine whether it stores a predetermined pattern indicating an unformatted state. If the pattern is found, metadata for the designated location are generated and returned. Otherwise, metadata are returned as initially read from the storage volume.

Just-in time formatting of file system metadata

An improved technique involves providing a pair of inode numbers in a directory entry for a file in a directory of a target file system. Along these lines, the target file system writes a first inode number and a second inode number to the directory entry for the file. While the client has not yet remounted its file system, that client may continue to use the file handle provided for the original file system. The data storage system continues to refer to a mapping table that cross-references initial inode numbers to target ones, at some cost in performance. Nevertheless, the client has an option to remount the file system at a time that is convenient to the client, so that the client may obtain a new handle. The client may subsequently use the new file handle to access the file.

Recovering performance of a file system post-migration

For enabling a fast reverse lookup of parent directories storing records of hard links to a specified regular file, a file system manager maintains a hard link database (HLDB) storing information for addressing hard links to the regular files having multiple hard links. The HLDB contains a corresponding HLDB file for each regular file having multiple hard links. In a preferred implementation, the inode of each regular file stores the addressing information for one hard link to the regular file, a corresponding HLDB file stores the addressing information for other hard links to the regular file, each HLDB file is a directory accessed by invoking file system manager directory access routines, and the HLDB has a directory tree providing a multi-level hash index for searching the HLDB given the inode number of a regular file.

Growing community of inventors

South Orange, NJ, United States of America

Morgan A Clark