Emc Corporation

Isilon Systems, Inc.

Qumulo, Inc.

Justin Husted

Neal Thomas Fachan

Aaron Passey

Peter John Godman

Eric Michael Lemar

Darren P Schack

Robert J Anderson

Alain Hamel

Max Laier

John Brazel

Kevin Ross O'Neill

Jeffrey Douglas Hughes

Thomas Scott Urban

Jonathan Mark Walton

Asif Arif Daud

Matthew D Fleming

David Richards

Nathan Edward Dire

Mohd Fahadullah

Graeme Williams

Rachel J Hestilow

Ronald Stuart Steinke

Dmitri Chmelev

A software and hardware facility persistently maintains metrics on directories at different levels within a tree of a filesystem. The facility hierarchically aggregates attributes of files contained by directories and stores them as metric values in each directory within a tree. The stored values represent summed or otherwise aggregated data from the descendant directories and files. The metric values represent aggregated attributes such as total space consumed by a directory and all the descendant directories and files, total number of files within a directory, total data blocks used by a directory and its descendant directories and data files, etc.

Filesystem hierarchical aggregate metrics

Embodiments are directed towards managing designations of a plurality of files across multiple clusters. Files may be designated as primary files or secondary files. Primary files at one cluster may be asynchronously replicated at another cluster as secondary files. Secondary files may be synchronously validated with remote primary files based on local file access requests. If a write request for a local secondary file is provided, then the local file may be designated as the primary file and the remote file may be designated as the secondary file. Local access may then be provided to the local primary file. If a read request for a local secondary file is provide, then local access may be provided to the local secondary file without changing file designations. In some embodiments, file designations may be preemptively changed based on a file change designation schedule for clusters at different geographic locations.

Geographic distribution of files

Implementations are provided herein for sending event notifications based on modifications to files and/or directories. When a file is read, modified or changed under the conditions for a registered event notification, a consistency point can be established. An impact list can then be determined based on the target. It can then be determined whether the version of event caches associated with the files and folders identified in the impact list match a global event cache version determined at the time of the consistency point. If the event caches are an old version, the system call can be restarted, and the event caches can be rebuilt to the current version. If the event caches of the impact list are current, the file system operation can be performed, and notifications can be sent based on the set of listeners identified within the current set of event caches.

Listener event consistency points

Embodiments are directed towards cloning files in a distributed file system. In response to a request to create file clones, a file cloning application can be employed to generate file clones based on a source file. The file cloning application can generate a new inode for each new file clone and at least one new shadow store. Data blocks from the source file can be transferred to the shadow store. The transferred data blocks can be replaced with reference blocks that point to the data blocks transferred to the shadow store. Likewise, additional reference blocks pointing to data blocks in the shadow store can be generated for the new file clone inodes. Further, the shadow store can maintain reference counters that track the number of reference blocks pointing to each shadow store block. Also, file clones can be generated based on a point-in-time snapshot of the source file.

File clones in a distributed file system

Various embodiments are directed towards enabling data writes utilizing a node cache and a logstore stored on a stable storage device. A client device may send data to a node for writing to a parent file. The node may cache the received data prior to writing the data to the parent file. Caching the received data may comprise adding the received data to a node cache and to a logstore. In one embodiment, the node cache may include a coalescer that combines data from a plurality of data writes from the client device to the node prior to writing to the parent file. In some embodiments, the logstore may be mirrored logstore on one or more stable storage devices. The parent file may periodically be updated from the node cache data. After the parent file is updated, the node cache data and the logstore data may be purged.

Recovery and flush of endurant cache

Endurant cache

In one embodiment, systems and methods are provided for nonlinear journaling. In one embodiment, groups of data designated for storage in a data storage unit are journaled into persistent storage. In one embodiment, the journal data is recorded nonlinearly. In one embodiment, a linked data structure records data and data descriptors in persistent storage.

Systems and methods for providing nonlinear journaling

In one embodiment, a user or client device is connected to a distributed file system comprised of one or more physical nodes. The data on each of the physical nodes store metadata about files and directories within the file system. Some of the embodiments permit a user to take a snapshot of data stored on the file system. The snapshot may include a single file, a single directory, a plurality of files within a directory, a plurality of directories, a path on the file system that includes nested files and subdirectories, or more than one path on the file system that each includes nested files and directories. In an embodiment, a snapshot tracking data structure is maintained for efficient creation and deletion of the snapshot.

Systems and methods for a snapshot of data

In some embodiments, storage devices, such as a storage drive or a storage node, in an array of storage devices may be reintroduced into the array of storage devices after a period of temporary unavailability without fully rebuilding the entire previously unavailable storage device.

Systems and methods for managing unavailable storage devices

Embodiments of the invention relate generally to incremental computing. Specifically, embodiments of the invention include systems and methods that provide for the concurrent processing of multiple, incremental changes to a data value while at the same time monitoring and/or enforcing threshold values for that data value. For example, a method is provided that implements domain quotas within a data storage system.

Systems and methods of quota accounting

In one embodiment, a user or client device is connected to a distributed file system comprised of one or more physical nodes. The data on each of the physical nodes store metadata about files and directories within the file system. Some of the embodiments permit a user to take a snapshot of data stored on the file system. The snapshot may include a single file, a single directory, a plurality of files within a directory, a plurality of directories, a path on the file system that includes nested files and subdirectories, or more than one path on the file system that each includes nested files and directories. According to one embodiment, the system stores representations of the snapshots that govern a file or directory and its descendants within metadata associated with that file or directory. In one embodiment, before modifying a file or directory, the system updates the representations of the governing snapshots by traversing the metadata of ancestors of the file or directory.

Embodiments of the invention relate generally to incremental computing. Specifically, embodiments of the invention include systems and methods that provide for the concurrent processing of multiple, incremental changes to a data value while at the same time monitoring and/or enforcing threshold values for that data value. For example, a method is provided that determines whether multiple incremental changes to a data field could pass a threshold based on keeping track of a possible value range for the data field.

Growing community of inventors

Seattle, WA, United States of America

Justin Husted