Emc IP Holding Company LLC

Ronald Steinke

Suraj Brahma Raju

Max Laier

Douglas Kilpatrick

Suraj Raju

Jonathan Mark Walton

Dharmesh Shah

Antony Richards

David Leimbach

Mark Ranger

Steven Hubbell

Tanuj Khurana

Raeanne Marks

Karthik Palaiappan

Facilitating checkpoint locks for distributed systems is provided herein. A system can comprise a processor and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations. The operations can comprise generating an ordered data structure that comprises respective identifications of a group of nodes granted a shared lock for a data file. The group of nodes can comprise a node having a respective identification of the respective identifications. The operations also can comprise, based on a determination that a checkpoint function associated with the shared lock has completed at a node of the group of nodes, removing the respective identification of the node from the ordered data structure. In an example, the shared lock can be a checkpoint application program interface. In another example, the group of nodes can be nodes arranged in a distributed computing hierarchy.

Facilitating checkpoint locks for distributed systems

Techniques for address mirroring of a file system journal are presented. A transactional file system update can correspond to a write operation to be performed at a first participant node of a data storage system. A journal transfer component can transfer the transactional file system update from an initiator node of the data storage system to a first journal associated with the first participant node and recovery information relating to the update to a second journal associated with a second participant node of the data storage system. The recovery information can be determined based on the update. To facilitate the update, a journal descriptor block can be generated and sent to the second participant node, such block comprising the recovery information, which can include transaction state, block addresses involved in the transaction, error correction-related information, or a mirror block address set for the block addresses where desired.

Address mirroring of a file system journal

Systems and methods facilitating fault tolerance for transaction mirroring are described herein. A method as described herein can include receiving a commit command for a data transaction from an initiator node of the system, wherein the data transaction is associated with a first failure domain, and wherein the commit command is directed to a primary participant node and a secondary participant node of the system; determining whether a response to the commit command has been received at the primary participant node from the secondary participant node in response to the receiving; and, in response to determining that the response to the commit command was not received at the primary participant node, indicating that the secondary participant node is invalid in a data store associated with a second failure domain that is distinct from the first failure domain.

Fault tolerance for transaction mirroring

Sharing a state between network computing devices is described. For instance, a first state update can be received from a second computing device via a network connection. Further, in a shared state file stored in a storage device coupled to the first computing device, a first portion of the shared state file is identified that is allocated to the second computing device. Further, based on the first state update, the first portion of the shared state file can be updated.

Maintaining a shared state among networked computing devices using a shared data store

The described technology is generally directed towards allocating adjacent file parts to different virtual cylinder groups mapped to storage devices in a storage array system. According to an embodiment, a system can comprise a processor and computer executable components that can comprise a file allocating component that can allocate a first part of a file to a first logical data block mapped to a first physical data block on a first storage device, and allocate a second part of the file to a second logical data block mapped to a second physical data block on a second storage device, the allocating being based on the second physical data block being on a different storage device than the first physical data block. The components can also comprise a storage device controller that can write the first and second parts of the file to the first and second physical data blocks.

Allocating adjacent file parts to different virtual cylinder groups mapped to physical storage devices

Implementations are provided herein for using a Node State Block ('NSB') to track the validity of various copies of the journal. The NSB can contain states local to the node and be implemented independent of the cluster state, and be readable at node boot time. The NSB can be mirrored to every storage drive within the node. The NSB can contain a generation number and a validity bit. Every time the NSB is updated, the generation number can be incremented. An update must be written to a quorum of drives to be considered committed. At mount time, the NSB is read from all the drives, and the copy with the highest generation number is used. If a quorum of drives cannot be read, the mount fails. Read quorum, or the quorum required at mount when reading the NSB across the set of storage drives requires a strict majority of the storage drives to have valid copies of the NSB. Write quorum, or the quorum required to update the NSB across all storage drives, requires a simple majority of drives.

Mirrored block with quorum set management for use in tracking valid mirrors of a journal

Implementations are provided herein for using inode revision numbers associated with a modified LIN and a set of Parent LINs to causally order transactions within a distributed file system. Any time an inode is changed, its inode revision number can be incremented by 1. When events within file system are processed causing an inode or a set of inodes to be modified, an event transaction log entry can made. The event transaction log entry can denote a description of the event, a set of modified inode and inode revision number pairs, and a set of parent inode and inode revision number pairs. Entries in the event transaction log can be used to build an inode map for each inode implicated in the event transaction log. The inode map can be used to build a set of direct causal dependencies for each transaction in the event transaction log. The set of direct causal dependencies can be used to generate a causal ordering of the transactions in the event transaction log, that in some implementations, can be made available to external services.

Causally ordering distributed file system events

Implementations are provided herein for a node among a cluster of nodes to maintain copies of its journal both locally and on a buddy node. Each copy of the journal can be associated with a validity bit and a clean bit that can be provided as inputs to a journal mirror state machine that can be used to maintain consistency between local copies of the journal and mirrored copies of the journal. The node can operate in a read-only mode or a read-write mode, whereby operating in read-only mode prevents changes to the local journal. The status of the mirror copy of the journal can be established in a link status depending on its availability. The journal can then transition between various states of the state machine upon triggering events that change the link status or the mode of the node. It can be appreciated that the transitioning among states of the state machine can provide crash-consistency for the filesystem during operation.

Method for tracking validity of journal copies to allow journal mirroring

Implementations are provided herein for using a distributed DNS cache that is distributed among nodes of a cluster of nodes operating as a distributed file system. A consistent hash can be used to determine a resolution node for a DNS request asking to resolve a specific FQDN. The inputs to the consistent hashing algorithm can be the FQDN, a set of available nodes, and a set of all possible nodes. By using a consistent hash, the process can reduce the sensitivity of the hashing algorithm output from changing when nodes added or removed as participants in the distributed cache service. As each node in the cluster of nodes can independently calculate a resolution node for specific FQDN, there is no need to send control messages between nodes, as each node only needs to be aware of the set of available nodes in the cluster of nodes.

Applying a consistent hash to a distributed domain name server cache

Implementations are provided herein for the use of multiple threads in concurrently restriping files by maintaining multiple cursors for a single file. The cursors can be stored and tracked on disk in a mirrored data structure ('MDS') extension block also referred to as a Cursor Extension Block. During a restripe operation, individual cursors stored in the delta-block portion of the MDS extension block will be updated using delta operations of a file system. The operating system kernel can maintain an in-memory version of the delta-block portion of the MDS extension block as well. The Cursor Extension Block associated with a file can store data representing the ending logical cluster number of the cursor, the current logical cluster number of the cursor, and a current cursor count for a file.

Multiple concurrent cursors for file repair

Implementations are provided herein for sub-block ranged locking on a journal block. A file system locking scheme can be adjusted to provide ranged locks on sub-blocks of a journal block. By locking sub-block ranges of a journal block, two independent operations on the same block that do not overlap on a sub-block range can operate independent of each other and be processed concurrently without requiring an exclusive lock over the entire block. Delta write locks, delta read locks, and shared locks can be established on a sub-block range level.

Growing community of inventors

Tacoma, WA, United States of America

Ronald Steinke