Dell Products Inc.

Murthy Mamidi

Charles J Hickey

Neerajkumar Nareshkumar Chourasia

Shashikanth Lakshmikantha

Geetha Shendge

George Mathew

Mukesh Sharma

Keyur B Desai

Srisailendra Yallapragada

Dhawal Bhagwat

Page keys are formed in a key-value store (KVS) storing metadata of a filesystem. Page keys identify pages of trees within which namespaces of logical partitions are held. Leaf pages include first and second full keys for each file. First full keys index inodes. Second full keys index filenames. Inode keys are formed in the KVS. Inode keys identify inodes of the files. Metadata is embedded into the page and inode keys such that each page key includes an identifier of a logical partition, an identifier of a snapshot taken of a namespace of the logical partition, and a page number to a page tree holding the namespace. Each inode key includes the logical partition identifier, the snapshot identifier, and an inode number of an inode to a file. Upon damage to the namespace, queries are conducted of the KVS and the namespace is recovered based on the query results.

Namespace recovery for key-value store (KVS)-persisted metadata

Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Continuous namespace verification for single-node filesystems

Embodiments for performing load balancing in a filesystem of a distributed network comprising worker nodes accessing shared resources that are controlled by distributed locks to grant and prevent access from clients, by assigning a range of directory entries to each worker node, and providing feedback from each worker node to a load balancer to expose an internal state of acquired resources and other parameters of each worker node. In response to periodic updates of the directory ranges to be collocated, the system maintains directory ranges in each worker so that complete subtrees of the directory tree fit in a single range assigned to a worker. Upon receipt of a client request to access a data asset, the load balancer selects the most optimum worker to satisfy the request based on the worker internal states and other parameters.

Intelligent load balancing in a distributed system

Continuous namespace verification for cluster filesystems

Adaptive namespace verification for single-node filesystems

Adaptive namespace verification for cluster filesystems

Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata for the file. If a legacy location in the metadata for the file includes any value which was stored after a most recent replication involving the file, the source node identifies a key for the legacy location in the metadata for the file. The source node adds the grouping tag as a value for the identified key. The source node sends the metadata for the file, including the identified key, to a destination node.

Source nodes for replication of tags in global scale systems

Upon completing replicating a file set from a source to destination, a snapshot taken of the file set is maintained at the source. The file set includes a namespace file having metadata and other files having content data. Verification is started on the file set replicated to the destination. While the verification is in-progress, a next replication cycle is started. Upon detecting corruption in the namespace file, the next replication cycle is paused and any changes to the file set are rolled back. The snapshot being maintained at the source is fetched. The namespace file having the corruption is replaced with the namespace file from the snapshot being maintained at the source. Other files in the file set having the content data are not replaced.

Fast recovery for replication corruptions

Systems and methods for reconstructing a namespace in response to a loss of data are described herein. To reconstruct the namespace, metadata super segments within a set of file containers may be located in response to a determination that original namespace metadata associated with files stored within the containers is unreadable. Content handles linking the super segment to metadata for each of the files may then be reconstructed from the super segments. Both inode numbers and M-tree identifiers are also extracted from the super segments for each of the files referenced by the super segments. Then, a plurality of U-trees is assembled by aggregating and linking the inode numbers of the files referenced within the super segments based on the M-tree identifiers. Finally, the recovered namespace metadata tree data structure for all of the files within the plurality of LP containers is assembled by aggregating and linking the U-trees.

File system namespace reconstruction

Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata. If the legacy location in the metadata includes any value which was stored after the most recent replication involving the file, and the size of the legacy location in the metadata was not changed after the most recent replication involving the file, the system adds the grouping tag to the legacy location in the metadata which includes any value which was stored after the most recent replication involving the file. The system sends the metadata, which includes the grouping tag and any value stored at the legacy location in the metadata, to a destination node.

Replication of tags in global scale systems

A destination node receives metadata for a file, via replication, from a source node. If a legacy location in the metadata for the file does not include any new value for a legacy location in local metadata for the file, then the destination node stores the metadata to a local data structure. If the legacy location in the metadata includes any new value for the legacy location in the local metadata, and the new value is any new grouping tag, then the destination node extracts the new grouping tag from the legacy location in the metadata. The destination node constructs a key based on the legacy location in the metadata and uses the constructed key to search for an alternative location in the metadata for the file, for the new grouping tag. The destination node stores the new grouping tag in the alternative location in the metadata.

Creating keys for replication of tags in global scale systems

A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata. If the legacy location in the metadata includes any value which was stored after the most recent replication involving the file, and the size of the legacy location in the metadata was changed after the most recent replication involving the file, then the system enables a destination node to store the grouping tag in the legacy location in the metadata by sending records comprising a request to replace a record stored at the legacy location in the metadata with a larger record, and combining the grouping tag with any value which changed the size of the metadata after a most recent replication involving the file.

Growing community of inventors

Queen Creek, AZ, United States of America

Murthy Mamidi