Emc IP Holding Company LLC

Emc Corporation

Dell Products Inc.

Alexander S Mathews

Jean-Pierre Bono

Rohit K Chawla

Soumyadeep Sen

Philippe Armangau

Ahsan Rashid

Kumari Bijayalaxmi Nanda

Walter Forrester

Marc A De Souter

Morgan A Clark

Dixitkumar Vishnubhai Patel

Michal Marko

Xianlong Liu

Ruiling Dou

William C Davenport

Pranit Sethi

Michael Scheer

Changyong Yu

Chang Yong Yu

Gyanesh Kumar Choudhary

Miles Aram De Forest

Feng Zhang

Sairam Veeraswamy

Yining Si

Hongliang Tang

Qi Mao

Yunfei Chen

Michael D Schouten

Henry Hao Fang

Bijayalaxmi Nanda

Yuetao Jiang

James P Ferrigno

Described is a technology that facilitates directory level snapshots at any level of a filesystem's namespace. A directory inode mapping file associates a first directory's virtual inode number(s) with real inode number(s). A snapshot of the first directory creates a second directory with a second directory inode mapping file copied from the first directory inode mapping file, thereby sharing the real inode data structure(s) of the first directory. In the event that one of the directories is to be modified, the directory real inode is split to provide a new directory real inode file for the to-be-modified directory, with the modification after the split.

Directory snapshots based on directory level inode virtualization

The technology described herein efficiently determines whether a real inode is shared among views, or owned. In-memory data structures include a view snapshot generation counter that is increased as a snapshot that generates a view is created, and an inode total weight. An in-memory virtual inode cache dataset for a filesystem object associated with the view is instantiated with the value of snapshot generation counter, sharing-related data based on the inode mapping file entry for the object, and an inode access weight. To determine whether the inode is shared (and needs to be split), such as on a write to the object, the in-memory data is evaluated. The real inode is shared if the generation counters are unequal, if the sharing-related data indicates sharing at an intermediate indirect block level, or indicates sharing at the inode level and the inode access weight is less than the inode total weight.

Determining a sharing relationship of a file in namespace snapshots

Described is hard link handling when a directory snapshot exists that includes the hard link's connected file object. A hard link is created by allocating a virtual inode number for the hard link, with the virtual inode number mapped to a real inode number that identifies a real inode of the file object; the hard link is assigned weight. A total weight associated with the real inode is increased by the hard link weight, and a hard link data store is updated with an entry for the hard link. Upon receiving data write request to the hard link, weight data determines that the file object is shared as a result of the snapshot; the hard link is disassociated from the real inode file, and associated with a new real inode number and new real inode of a new file object. The data is written based on the new real inode.

Hard link handling with directory snapshots

A method, computer program product, and computer system for maintaining a back pointer from a physical layer block (PLB) to a virtual layer block (VLB) in a multi-level hierarchical file system. A generation number may be maintained in the VLB, wherein the generation number may indicate when data is moved from the PLB to another PLB. An object may be reconstructed in the multi-level hierarchical file system based upon, at least in part, at least one of the back pointer and the generation number.

System and method for inline recovery of a file system resource allocator

Technology described herein can perform deletion of a snapshot or portion thereof. In an embodiment, a system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise, to delete a snapshot, or a portion of a snapshot, of a real filesystem, reading an inode mapping file (IMF) of the snapshot that indexes a virtual inode number (VIN) corresponding to a real inode. The operations further can comprise identifying the real inode of the snapshot referenced by the VIN, identifying a file object corresponding to the real inode, and deleting the file object from the snapshot.

Deletion of snapshot of filesystem

A method for file system reorganization in the presence of inline compression includes obtaining a virtual block pointer for an original compressed segment to be reorganized, the original compressed segment comprising compressed allocation units of data stored in a storage system, wherein the virtual block pointer comprises an extent list identifying the compressed allocation units in the original compressed segment and a pointer to where the original compressed segment is stored; copying only the referenced compressed allocation units in the original compressed segment to a new compressed segment in a substantially contiguous manner; updating the extent list to identify the referenced compressed allocation units in the new compressed segment, and the pointer to where the new compressed segment is stored; and freeing the original compressed segment.

File system reorganization in the presence of inline compression

An apparatus comprises a processing device configured to generate a tree structure characterizing relationships between storage objects in a storage system represented as logical page nodes specifying respective logical page addresses, arrays of pointers to other logical page addresses, snapshot group identifiers, and logical extent offsets. The processing device is also configured to traverse the generated tree structure to identify (i) a given logical page node specifying a given logical page address, snapshot group identifier and logical extent offset from a query and (ii) other ones of the logical page nodes that specify the given snapshot group identifier and logical extent offset and comprise a pointer to the given logical page address in its associated array of pointers. The processing device is further configured to provide a response to the query specifying the given logical page node and the identified other ones of the logical page nodes.

Finding storage objects of a snapshot group pointing to a logical page in a logical address space of a storage system

Techniques for performing space accounting for volume families. The techniques include maintaining two counters for each volume family, including a first counter that tracks a first amount of physical space allocated to volumes in the volume family based on the number of pages written to the volumes, and a second counter that tracks a second amount of physical space owned by the volume family, and maintaining a third counter for each volume in each branch of the volume family that tracks a number of pages among a total number of pages written to the volume that were overwritten in the immediate sequential next volume in the same branch as the volume. By maintaining, for each volume family, the first counter and the second counter, and, for each volume in the volume family, the third counter, space accounting metrics can be obtained that allow data storage activities to be performed efficiently.

Mechanisms for performing accurate space accounting for volume families

A method of repairing an indirect addressing structure of a file system damaged by corruption of a virtual data block (VDB) mapping data stored in corresponding physical data blocks (PDBs) includes scanning PDB descriptors to identify PDBs storing data mapped by the corrupted VDB, where each identified PDB includes a set of PDB entries each having a backward pointer identifying a corresponding VDB entry of a corresponding VDB. The identified PDBs are scanned to identify PDB entries whose backward pointers refer to VDB entries of the corrupted VDB, then a replacement VDB is created by (1) for each of the identified PDB entries, recreating a corresponding VDB entry including a forward pointer to the identified PDB entry, and (2) incorporating the recreated VDB entries into the replacement VDB. The replacement VDB is then incorporated into the indirect addressing structure in place of the corrupted VDB.

Data storage system with repair of virtual data blocks of internal file system

A technique for repairing an indirect addressing structure of a file system damaged by corruption of a mid-level mapping (MID) page includes scanning selected leaf pages to identify leaf pages associated with the corrupted MID page, then recreating the MID page by recreating pointers to the identified leaf pages. The scanning includes (1) based on an association of groups of leaf pages with corresponding sets of families of storage objects, scanning the leaf pages of only those groups of leaf pages associated with the family of storage objects for the corrupted MID page. The scanning further includes (2) performing a two-pass process including first identifying all leaf pages for the logical offset range of the corrupted MID page and then pruning those identified leaf pages that are reachable via non-corrupted MID pages, yielding the leaf pages for the corrupted MID page only, usable to recreate the corrupted MID page.

Data storage system with repair of mid-level mapping blocks of internal file system

An in-line (or foreground) approach to obtaining contiguous ranges of free space in a file system of a data storage system that can select windows having blocks suitable for relocation at a time when one or more blocks within the respective windows are freed or de-allocated. By providing the in-line or foreground approach to obtaining contiguous ranges of free space in a file system, a more efficient determination of windows having blocks suitable for relocation can be achieved, thereby conserving processing resources of the data storage system.

Inline coalescing of file system free space

A data storage system manages use of a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients. A choose-and-mark-slices operation scans the file system to identify a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset having a size exceeding a per-iteration limit observed by a separate evacuate-slices operation. The subset is recorded in an in-memory structure. Each iteration of the evacuate-slices operation (1) examines the in-memory structure to select slices for evacuation, no more than the per-iteration limit being selected, (2) moves data from the selected slices to the remaining slices of the set, and (3) subsequently (i) updates metadata of the file system to reflect the moving of the data and (ii) de-provisions the selected slices from the file-system to return them to the pool for subsequent re-provisioning.

Data storage system using in-memory structure for reclaiming space from internal file system to pool storage

A method is used in managing data relocations in storage systems. A request is received to relocate a file system block of a file of a file system. The file includes a set of file system blocks. A determination is made as to whether the file system block has been identified for performing an operation on the file system block. The operation impacts the determination of whether to skip relocating of the file system block. Based on the determination, updating mapping information of the file system block is avoided upon relocation of the file system block.

Managing data relocations in storage systems

A method includes (a) while processing data storage requests from users directed to filesystems of a plurality of filesystems served from a data storage system, receiving a request to execute a filesystem reorganization task on a filesystem of the plurality of filesystems, (b) in response to receiving the request, checking whether the filesystem reorganization task has both a global reorganization token and a filesystem-specific reorganization token, the filesystem-specific reorganization token being specific to the filesystem on which the filesystem reorganization task has been requested to execute, and (c) in response to determining that the filesystem reorganization task does have both the global reorganization token and the filesystem-specific reorganization token, executing the filesystem reorganization task to move a data block from one location within the filesystem to another location within the filesystem. Other embodiments are directed to corresponding apparatuses, computer program products, and systems for performing similar methods.

Techniques for efficiently performing filesystem reorganization tasks for different filesystems

Techniques for tracking pages that are unshared among snapshot volumes and their parent volume in a volume family. The techniques include maintaining a first counter that tracks a first number ('SnapOwnedCount') of pages among a total number of pages written to a volume that were overwritten in the immediate sequential next volume in the same branch as the volume, and a second counter that tracks a second number ('UnsharedWithParentCount') of pages among the total number of pages written to the volume that were overwritten in any sequential next volume in the same branch as the volume. By maintaining, for each volume in a volume family, a first counter and a second counter for tracking the SnapOwnedCount and the UnsharedWithParentCount, respectively, data storage recovery can be performed more effectively, allowing selected volumes and/or portions thereof to be deleted while maintaining accurate space accounting information.

Growing community of inventors

Morganville, NJ, United States of America

Alexander S Mathews