Dell Products Inc.

Emc IP Holding Company LLC

Oliver Chung

Evgeny Popovich

Austin Voecks

Smitha Jaya Shankar

Jonathan Mark Walton

Michael Frank

The described technology is generally directed towards replicating a filesystem namespace from a source system such as a source node cluster to a target system such as a target node cluster. Directory transfer tasks processed at the source are sent to the target build a replica of a hierarchical namespace on a target cluster, in which the namespace contains the directories and files, with the file content temporarily absent. Files transfer tasks processed at the source are sent to the target to write the content into the files, generally after namespace creation is complete. Directory transfer is recursive, as one directory transfer task creates another directory transfer task for each of its child directories, and can be concurrent and sent as bulk transfer requests for efficiency in reducing overall latency overhead. File content transfer can transfer the content of multiple small files in a single transfer task request.

Avoiding directory contention in distributed filesystem replication

The described technology is generally directed towards efficient detection of data changes between different data representations, to enable replication of the detected data changes. A process is presented that compares different 'snapshot' representations of data, wherein the data may have changed between the different snapshots. The process can detect changed files a well as namespace changes that occur when a file or directory is deleted, added, or moved to a different location.

Efficient detection of data changes between different data representations

The described technology is generally directed towards resolving namespace collisions on a target system when applying incremental changes (e.g., between snapshots) as part of an incremental replication from a source system. Detected changes to a filesystem namespace include file link, directory move and file and directory unlink operations. The changes are processed to recognize and overcome failures and other issues, such as to not process an unlink before a corresponding link, which can otherwise result in accidental deletion of a file. Deferred operations are maintained for later processing, including deferred directory moves for directory rename failures, deferred unlink operations and name collision fixes. After initial processing, the deferred move list is iteratively processed to overcome directory rename failures, the deferred unlink list is processed after performing corresponding link operations, and the name fix list is processed to rename previously colliding names.

Resolving namespace collisions during filesystem incremental replication

Incremental replication between foreign system dataset stores is described herein. A method as described herein can include connecting to a target storage system to facilitate an initial transfer of a dataset from the system to the target storage system, wherein the system is distinct from the target storage system; in response to determining that a first version of the dataset is present on the target storage system, comparing the first version of the dataset to a second version of the dataset stored on the system; and, in response to determining that the second version of the dataset is more recent than the first version of the dataset, transferring a differential between the first version of the dataset and the second version of the dataset to the target storage system, the differential comprising less than all of the second version of the dataset.

Growing community of inventors

Bellevue, WA, United States of America

Oliver Chung