Qumulo, Inc.

Netapp, Inc.

Kevin David Jamieson

Sihang Su

Michael Anthony Chmiel

Aleksei Martynov

Christopher Charles Harward

Kevin Ross O'Neill

Nicholas John Carter

Junjie Liang

Sasha Spielberg Friedrich

Neal Thomas Fachan

Logan Jennings

Gobikrishnan Sundharraj

Porter Michael Smith

Daniel Shvarts

Landon John Meernik

Adam Mitha

Pavel Shilovskiy

Colin Wallace

Embodiments are directed to bandwidth management in distributed file systems. A request stored in a queue may be dequeued. A work credit value may be determined based on a duration since a last job and a work rate associated with the file system such that the work credit value conforms to a maximum threshold value. A job may be initialized based on the dequeued request such that the work credit value may be reduced based on a size of the job. In response to the reduced work credit value being a negative value further actions may be performed, including: determining a time delay based on the reduced work credit value, the work rate, and the size of the job; delaying an execution of the job until the expiry of the time delay such that upon expiry of the time delay, the job is executed to satisfy the request.

Bandwidth management in distributed file systems

Embodiments are directed to managing data in a file system. A source bucket that contains a plurality of objects on a cloud computing environment may be determined. The source bucket may be associated with a target directory in the file system based on an integration relationship. Object keys from the object store may be provided based on a query and filter rules such that each object key corresponds to an object in the source bucket that conforms to the filter rules. Source objects in the source bucket may be determined based on the object keys such that each source object corresponds to one of the object keys. Target files that correspond to the source objects may be determined based on the integration relationship. In response to determining changed target files or changed source objects, copying the changed source objects or the changed target files to the target directory.

Integrating distributed file systems with object stores

Embodiments are directed to managing data in a file system over a network. A source file system that includes a plurality of objects may be provided. A replication job that copies each object associated with a source replication snapshot to a target file system may be executed. The replication job may be associated with a job identifier. Recovery point information that includes the job identifier, a source snapshot number that corresponds to the source replication snapshot, a target snapshot number that corresponds to a target replication snapshot may be generated. The recovery point information may be stored on the source file system and a copy of the recovery point information may be stored on the target file system. The recovery point information or the copy of the recovery point information may be employed to recover from errors detected during execution of a next replication job.

Recovery checkpoints for distributed file systems

Embodiments are directed to traversing a file system to determine file system objects to copy to an object store. In response to visiting a document object in the file system, performing further actions, including: determining a hierarchical file path of the document object that corresponds to a location in the file system based on the file system objects that are ancestor file system objects of the document object; generating an object key for the document object that encodes the hierarchical file path such that each portion of the object key corresponds to an ancestor file system object; copying the document object and the object key to the object store such that the document object may be stored in the object store as an object store object and such that the object store object may be indexed using an unordered index based on the object key; or the like.

Replicating files in distributed file systems using object-based data storage

Embodiments are directed to a replication engine that provides a root node for a file that may be associated with a replication snapshot where the file is comprised of the root node, internal nodes, and data nodes. The replication engine may start at the root node and traverse the file system. If a visited internal node has an epoch value that matches the epoch, the replication engine may continue the traversal by visiting a next unvisited adjacent internal node. If the visited internal node has an epoch value that mismatches the epoch, the replication engine may backtrack to a nearest adjacent internal node that matches the epoch. And, in response to visiting a data node the replication engine may be arranged to determine blocks that are associated with the data node and communicate a copy of the blocks to a target file system.

Data replication in distributed file systems

Embodiments are directed to managing data in a file system. A source file system may be mirrored on a target file system by executing one or more replication jobs. A root object of the source file system may be associated with a current read-only identifier corresponds to a current update epoch of the source file system. A replication job that copies objects from the source file system to the target file system may be executed. A write request directed to the source file system may be received. Write objects associated with the write request may be determined The source file system may be traversed upwards from the write objects. A read-only identifier associated with each visited object may be determined based on the read-only identifier matching the current read-only identifier. The write request may be enabled based on the mismatch of the read-only identifier and current read-only identifier.

Managing file system state during replication jobs

Embodiments are directed to a file system engine that provides a file system with parent objects associated with child objects. The file system engine provides a replication snapshot associated with an epoch of the file system such that each child object modified during the epoch and each associated parent object is associated with the replication snapshot. A root object of a portion of the file system included in the replication snapshot may be provided. A replication engine may traverse the portion of file system starting from the root object such that the parent objects that are not associated with the replication snapshot are omitted from the traversal. The replication engine may determine replication objects based on the traversal such that each replication object was modified during the epoch. Then the replication engine may execute a replication job that copies replication objects to a target file system.

Replicating file system objects in distributed file systems

A machine implemented method and system for allocating a storage volume from among a plurality of storage volumes for storing information in a storage system is provided. The storage system maintains a plurality of groups for segregating the plurality of storage volumes based on storage volume utilization at any given time. A storage volume, when available, is selected from a group that was previously used to select another storage volume. After an end of the group has been reached for allocating the other storage volume, and a threshold value for using the same group to allocate storage volumes has been reached, then another group is used to select the next storage volume.

Growing community of inventors

Vancouver, Canada

Kevin David Jamieson