Dell Products Inc.

Emc IP Holding Company LLC

Mohammed Asher

Ramesh Doddaiah

Sandeep Chandrashekhara

Rong Yu

Mohammed Aamir Vt

Michael J Scharland

Malak Alshawabkeh

Deepak Vokaliga

Michael E Specht

Jeremy O'Hare

Jiahui Wang

Lixin Pang

Richard Goodwill

Mark Hakstead

Aamir Mohammed Vt

Aamir Mohammed

For a source device with an activated snapshot, time series-based prediction of source device write IO operations is implemented, on a per-extent basis, to predict a subset of the source device extents that are likely to be hot (receive write IO operations) during an upcoming time window. Snapshot tracks corresponding to tracks of the predicted hot extents are pre-deduplicated, to accelerate write IO operations on the source device. In instances where the time series-based prediction correctly predicts write IO operations on tracks of an extent, and the tracks of the extent of the source device are pre-deduplicated on the snapshot, it is possible to implement the write IO operations as a redirect on write operation, without first replicating the original track of source data for use by the snapshot. Write IO operations on tracks that are not pre-deduplicated are implemented as copy on write operations.

Time series forecasting for pre-deduplication of snapshots

Metadata page prefetch processing for incoming IO operations is provided to increase storage system performance by reducing the frequency of metadata page miss events during IO processing. When an IO is received at a storage system, the IO is placed in an IO queue to be scheduled for processing by an IO processing thread. A metadata page prefetch thread reads the LBA address of the IO and determines whether all of the metadata page(s) that will be needed by the IO processing thread are contained in IO thread metadata resources. In response to a determination that one or more of the required metadata pages are not contained in IO thread metadata resources, the metadata page prefetch thread instructs a MDP thread to move the required metadata page(s) from metadata storage to IO thread metadata resources. The IO processing thread then implements the IO operation using the prefetched metadata.

Metadata page prefetch processing for incoming IO operations

In asynchronous remote replication, write IOs are accumulated in capture cycles and sent to a remote storage system in transmit cycles. In order to cause metadata cache hits at the remote storage system, write IO data and associated metadata hints such as logical block addresses being updated are sent in successive cycles. The metadata hints, which are received at the remote storage system before the corresponding write IO data, are used to prefetch metadata associated with the logical block addresses being updated to replicate the write IO.

Metadata pre-paging on asynchronous remote replication to improve remote array performance

Selective packing of small block write operations is implemented prior to compression, to improve compression efficiency and hence reduce bandwidth requirements of a Remote Data Replication (RDR) facility. Compression characteristics of write IO operations are forecast, and write IO operations with similar forecast compression characteristics are pooled. Write IO operations are also grouped according to extent, device, and storage group. Write operations from a given compression pool are then preferentially selected from the extent-level grouping, next from the device-level grouping, and then from the SG-level grouping, to create an IO package. The IO package is then compressed and transmitted on the RDR facility. By creating an IO package prior to compression, it is possible to achieve greater compression than would be possible if each individual write IO operation were to be individually compressed to thereby reduce network bandwidth of the RDR facility.

Selective packing of small block write operations prior to compression

One or more aspects of the present disclosure relate to performant destaging of write pending (WP) data to disk. In embodiments, an input/output (IO) workload is received at a storage array. Additionally, the IO workload can include an IO request with a random write request. Further, a write destage context for write-pending (WP) data corresponding to the random write request can be generated by a data services engine of the storage array. In addition, using the write destage context, a disk adapter (DA), at a backend of the storage array, is enabled to destage write-pending (WP) data without reading from a target write location of the random write request on a storage device on the storage array.

Performant write pending data destaging

A remote data replication facility includes a primary storage array and a backup storage array, on which tracks of data are replicated from the primary storage array to the backup storage array as they are received by the primary storage array. Remote data verification is implemented on the remote data replication facility by comparing track fingerprints, track temporal write metadata, and track spatial write metadata, for a given track on the primary storage array, with corresponding track fingerprints, track temporal write metadata, and track spatial write metadata, for the given track on the backup storage array. If any difference is determined in the combination of track fingerprints, track temporal write metadata, track spatial write metadata, for a given track, the integrity of the data at the backup storage array is not verified for the track.

Remote data integrity verification on a remote data replication facility

The present disclosure relates to applying a data deduplication layer on top of an asynchronous remote replication protocol. In embodiments, a remote replication group including one or more of a storage array's logical unit numbers (LUNs) can be established. Further, at a remote system, —the remote replication group can be remotely replicated on a per LUN basis using a deduplication fingerprint of each LUN's data tracks corresponding to the remote replication group.

Asynchronous remote replication data deduplication layer

The meta data containing count and key fields of CKD records are reversibly decoupled from the user data of the data field so that the data can be deduplicated. Multiple CKD records may be coalesced into a larger size CKD track. The coalesced meta data is compressed and stored in a CKD hash table. The user data is hashed, and the hash is used as a hash key that is associated with the compressed meta data in the CKD hash table. When the hash of user data associated with a CKD write IO matches the hash key of an existing entry in the table, data duplication is indicated. The compressed meta data is added to the entry and the user data is deduplicated by creating storage system meta data that points to the pre-existing copy of the user data. The storage system metadata includes unique information that enables the corresponding compressed metadata to be subsequently located in the hash table to reassemble the CKD records.

Inline deduplication for CKD using hash table for CKD track meta data

Embodiments of the present disclosure relate to throttling processing threads of a storage device. One or more input/output (I/O) workloads of a storage device can be monitored. One or more resources consumed by each thread of each storage device component to process each operation included in a workload can be analyzed. Based on the analysis, consumption of each resource consumed by each thread can be controlled.

Throttling processing threads

In a data storage system in which a first storage array and a second storage array maintain first and second replicas of a production volume, the replicas are made discoverable and accessible while inconsistent. Each storage array maintains an invalid track list of inconsistencies. Initially, all tracks are marked as invalid. While background synchronization is eliminating inconsistencies, accesses to invalid tracks are resolved by exchanging data associated with IOs and updating the invalid track lists based on IO bias and other factors.

Growing community of inventors

Bengaluru, India

Mohammed Asher