Netapp, Inc.

Joseph Brown

Ellard Roush

Sangramsinh Pandurang Pawar

Dennis Dalessandro

Ananthan Subramanian

Ram Kesavan

Rohit Shankar Singh

Abdul Basit

Rajesh Rajaraman

William D Dallas

Houze Xu

Per Olov Wahlstrom

Hiroshi Ishii

John David McA'Nulty

Parag Sarfare

Brian Naylor

James Robert Morefield

Manish Bhuwania

Sally Wahba

Michael J Noeth

Snehal Kamble

Joanne Mikkelson

Javier Tsuyoshi Takimoto

Michael Scott Ryan

Satish Cherbrolu

Systems and methods for making use of non-persistent storage as the journaling storage media for a virtual storage system are provided. According to one embodiment, in order to meet the needs of Extreme Low Latency Workloads while also seeking to provide predictable performance and the lowest possible latency, ephemeral storage of the virtual storage system is used to preserve state information (e.g., in the form of boot arguments and an operation log journal) across a host failure recovery scenario in which the virtual storage system is expected to be redeployed within a compute instance brought up by a cloud environment of a hyperscaler on the same host, thereby providing improved data durability (fewer host failure scenarios that result in lost data) as compared to the use of ephemeral memory of the compute instance and lower write latency than the use of persistent storage provided by the cloud environment.

Using ephemeral storage as backing storage for journaling by a virtual storage system

Systems and methods for multiple device consumption of shared namespaces of ephemeral storage devices by a consumer of a virtual storage system are provided. In an example, multiple namespaces of respective ephemeral storage devices are shared among multiple of consumers of a virtual storage system by creating multiple partitions within each of the namespaces for use by respective consumers of the multiple consumers. Corresponding partitions of respective shared namespace may then be treated as a stripe set to facilitate multiple device consumption for a subsystem (e.g., operation log journaling) of the virtual storage system by striping data associated with input/output (I/O) requests of a consumer (e.g., a journaling driver) across one or more stripe units of one or more stripes within the stripe set.

Facilitating multiple device consumption of shared namespaces of ephmermal storage devices by a consumer of a virtual storage device

Methods, non-transitory computer readable media, and computing devices that execute a storage operation, without journaling the storage operation in a log, and withhold from a file system a list of freed inodes including an indication of an inode freed as a result of the execution of the storage operation. A consistency point operation is then initiated that retrieves storage operations logged as journal entries in the log and commits a result of each of the storage operations to data storage devices. A list of available inodes is updated based on the list of freed inodes, when the consistency point operation is determined to be complete. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations following a recovery.

Methods for improving journal performance in storage networks and devices thereof

A method, non-transitory computer readable medium and storage server computing device that determines when a generated storage operation corresponds with one of a set of predefined storage operations. The storage operation is executed and a result of the execution of the storage operation is withheld from a file system, when the determining indicates that the storage operation corresponds with one of the set of predefined storage operations. A determination is made when a consistency point operation has completed. The result of the execution of the storage operation is presented to the file system, when the determining indicates that a consistency point operation has completed. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations in a journal following a recovery.

Technology is disclosed for improving performance during playback of logged data storage operations. The technology can monitor a log to which data storage operations are written before data is committed to a volume; determine counts of various types of data storage operations; and when the counts exceed a specified threshold, cause the data storage operations to be committed to the volume. Some data storage operations can be coalesced during playback to further improve performance.

Performance during playback of logged data storage operations

A method, non-transitory computer readable medium, and source storage server computing device that mirrors dirty data to a destination storage server. The dirty data corresponds to storage operations not yet committed to one or more data storage devices of an aggregate to be transitioned. Ownership of the data storage devices is released. Teardown processing for the aggregate is performed subsequent to releasing ownership of the data storage devices. With this technology, the flushing or committing of dirty data to data storage devices is performed by a destination storage server after a data container is transitioned to the destination storage server. Additionally, non-critical teardown processing is also performed after a data container or aggregate is transitioned to the destination storage server. Accordingly, the duration of planned data container transitions can advantageously be reduced and aggregates can be brought online by a destination storage server for data access by applications more quickly.

Methods for facilitating planned data container transitions and devices thereof

A method, device, and non-transitory computer readable medium for mirroring data, comprising, selecting, based on a plurality of data attributes, a portion of local data in a local storage device for mirroring to a remote storage device and copying the selected portion of the local data to at least one cache memory of the remote storage device. Next a determination of when a failover event has occurred in the local storage device is made, wherein the failover event comprises an event in which the local data in the local storage device is inaccessible to a client computing device when the client computing device attempts to access the local data from the local storage device. A copy of the local data from the cache memory in the remote storage device is retrieved when the failover event is determined to have occurred.

Methods for flexible data-mirroring to improve storage performance during mobility events and devices thereof

A method, non-transitory computer readable medium and storage server computing device that allocates monotonically increasing sequence numbers to storage operations based on an order of the storage operations corresponding entries in a log. The storage operations are sequenced including by identifying a subset of the storage operations that are parallelizable. A file system data structure is allocated and populated with data associated with one of the storage operations. A determination is made when a rollback is detected based on whether a replay ledger index field of the file system data structure comprises one of the sequence numbers that is greater than another one of the sequence number corresponding to the one of the storage operations. The another one of the sequence numbers is inserted into the replay ledger index field, when the determining indicates that a rollback is not detected.

Methods for detecting out-of-order sequencing during journal recovery and devices thereof

Technology is disclosed for improving performance during playback of logged data storage operations. The technology can monitor a log to which data storage operations are written before data is committed to a data storage device or a volume; determine counts of various types of data storage operations; and when the counts exceed a specified threshold, cause the data storage operations to be committed to the data storage device or the volume. Some data storage operations can be coalesced during playback to further improve performance.

Techniques for detecting data loss during site switchover are disclosed. An example method includes storing at NVRAM of a first node a plurality of operations of a second node, the first and second nodes being disaster recovery partners. The method also includes during a switchover from the second node to the first node, receiving an indication of a first number of operations yet to be completed. The method further includes comparing the first number to a second number of operations in the plurality of operations stored at the NVRAM of the first node. The method also includes in response to the comparing, determining whether at least one operation is missing from the plurality of operations stored in the NVRAM of the first node. The method further includes in response to determining that at least one operation is missing, failing at least one volume.

Growing community of inventors

Raleigh, NC, United States of America

Joseph Brown