Dell Products Inc.

Emc IP Holding Company LLC

Ramesh Doddaiah

Owen Martin

Malak Alshawabkeh

Sandeep Chandrashekhara

Mohammed Asher

Arieh Don

Sanjib Mallick

Rong Yu

Deeder M Aurongzeb

Jaeyoo Jung

Malathi Ramakrishnan

John Krasner

Kaustubh Sahasrabudhe

Peng Wu

Michael J Scharland

Jeremy O'Hare

Sweetesh Singh

Anoop Raghunathan

Earl Medeiros

Venkata Ippatapu

Mohammed Asher Vt

Mohammed Aamir Vt

Vinay G Rao

Peng Yin

John Creed

Lixin Pang

Kaustubh S Sahasrabudhe

Shao Hu

Steven John Ivester

Jingtong Liu

Benjamin A F Randolph

Richard Trabing

Evan Barry

Steve Lathrop

Richard Goodwill

Mark Hakstead

Venkata Khambam

Aamir Mohammed Vt

Aamir Mohammed

G Vinay Rao

Bernard A Mulligan, Iii

Aspects of the present disclosure relate to tuning resource allocations based on a storage array feature's impact on the array's global performance. In embodiments, one or more input/output (IO) features used by a storage array to process one or more IO workloads are determined. Additionally, each IO feature's impact for processing the IO workload within a threshold performance requirement can be determined. Further, at least one IO feature's resource allocation can be tuned based on its IO workload processing impact.

Storage array resource allocation based on feature sensitivities

For a source device with an activated snapshot, time series-based prediction of source device write IO operations is implemented, on a per-extent basis, to predict a subset of the source device extents that are likely to be hot (receive write IO operations) during an upcoming time window. Snapshot tracks corresponding to tracks of the predicted hot extents are pre-deduplicated, to accelerate write IO operations on the source device. In instances where the time series-based prediction correctly predicts write IO operations on tracks of an extent, and the tracks of the extent of the source device are pre-deduplicated on the snapshot, it is possible to implement the write IO operations as a redirect on write operation, without first replicating the original track of source data for use by the snapshot. Write IO operations on tracks that are not pre-deduplicated are implemented as copy on write operations.

Time series forecasting for pre-deduplication of snapshots

In an active-active replication environment in which sequential read IOs are distributed across storage array ports and replicas, a first read data size controls selection of a different path to a currently targeted replica to distribute sequential reads across storage system ports and a second read data size controls when a different replica is selected to distribute the sequential reads across storage systems. The storage arrays that maintain the replicas are phase-coordinated to pre-fetch only sequential data that will be accessed in the near future from their local replica. For example, a first storage array prefetches sequential data from a first replica in chunks corresponding to the first data size until sequential data corresponding to the second data size has been prefetched, skips sequential addresses corresponding to the second data size, and prefetches sequential data corresponding to the first data size that is next in sequence relative to the skipped sequential addresses.

Enhancing array prefetch for sequential IOs in active-active remote replication

A storage system is configured with pools of processor cores. Each pool corresponds uniquely to one of the supported service levels of the storage system. In a dynamic time series forecast adjustment mode, processor cores within each pool run at a clock speed that is statically defined for the service level corresponding to the respective pool and core affiliations with pools are dynamically adjusted based on modelled data access latency. During a scheduled event, an event guided core matrix profile overrides the time series forecast adjusted core matrix profile. The event guided core matrix profile includes core clock speeds and pool affiliations, thereby enabling rapid reconfiguration.

Customer informed composable core matrix for sustainable service levels

One or more aspects of the present disclosure relate to distributing write destaging amongst a storage array's boards. In embodiments, an input/output (IO) workload is received at a storage array. In addition, a write IO request in the IO workload is directed to a target board of a plurality of boards of the storage array based on one or more performance-related hyperparameters of each board. Further, write data of the write IO request to a storage device is destaged by the target board.

Distributed write destaging

Autonomous power control is provided on a storage system to enable service level maximum response time controls to be achieved, while also minimizing power consumption, by enabling service level minimum response times to be specified and enforced. A workload/CPU clock speed model is created for the storage system correlating a maximum number of IOPS that the storage system can process for different CPU clock speeds for each workload type. A service level agreement specifies a maximum storage system response time. A storage system minimum response time is also specified, that is used to identify a target CPU clock speed for the workload type being provided by a host. The CPU clock speed is lowered to reduce energy consumption by the storage system toward the target CPU clock speed, and storage system performance is monitored to ensure that the storage system complies with the storage system maximum response time.

Autonomous power control based on service level compliance

Host IO write operations on a given storage group are grouped into cycles of a write destage pipeline for the storage group. Host writes are collected during a capture cycle, while host writes from a previous capture cycle are analyzed and destaged during an apply cycle. After the previous apply cycle has completed, a cycle switch occurs and the current capture cycle becomes the new apply cycle. Anomaly detection is implemented before any IO write operations are destaged during the apply cycle, and if an anomaly is detected the write destage is stopped. The entire apply cycle is either destaged to disk, or is discarded, depending on whether the anomaly is confirmed. By maintaining the order of writes across different cycles, which act as consistency points, it is possible to implement ransomware protection in real time on host write operations, while using ordered write destage to maintain data consistency.

Data protection by delayed ordered write destage operations

A shared global memory segmentation conversion process optimizes conversion of shared Global Memory (GM) in connection with changes to a global memory segmentation policy. When a policy region is to be reduced in size, the GM banks of the policy region are ranked and one or more of the lowest performing GM banks in the policy region are selected to be moved to the other policy region. Similarly, within each policy region, IO pools, used to create slots of particular sizes, that are being reduced in size are determined, and the GM banks of the IO pools are ranked. One or more of the lowest performing GM banks are selected and moved to an IO pool that is being increased in size. This process iterates, re-ranking the remaining GM banks in the policy region/IO pools, until the correct allocation of GM policy regions and IO pools is achieved.

Global memory segmentation conversion

Metadata page prefetch processing for incoming IO operations is provided to increase storage system performance by reducing the frequency of metadata page miss events during IO processing. When an IO is received at a storage system, the IO is placed in an IO queue to be scheduled for processing by an IO processing thread. A metadata page prefetch thread reads the LBA address of the IO and determines whether all of the metadata page(s) that will be needed by the IO processing thread are contained in IO thread metadata resources. In response to a determination that one or more of the required metadata pages are not contained in IO thread metadata resources, the metadata page prefetch thread instructs a MDP thread to move the required metadata page(s) from metadata storage to IO thread metadata resources. The IO processing thread then implements the IO operation using the prefetched metadata.

Metadata page prefetch processing for incoming IO operations

Streaming machine telemetry (SMT) event counters are placed in critical code paths of software executing on a storage system. Each monitoring interval the values of the SMT counters are reported. When a critical error occurs on the storage system, a time series set of SMT counters from a set of previous monitoring intervals is labeled with the error type and used as a training example for a learning process. The learning process is trained to learn to learn recursions between time series sets of SMT counter values and labeled error types. Once trained, a checkpoint of the learning model is deployed as an inference model and used to predict the likely occurrence of errors before the errors occur. Predicted errors are logged into a proactive service request queue, and feedback related to the predicted errors are used as feedback to continue training the learning process.

Proactive service requests for forecast storage system failures

In a storage system in which processor cores are exclusively allocated to run process threads of individual emulations, the allocations of cores to emulations are dynamically reconfigured based on forecasted workload. A workload configuration model is created by testing different core allocation permutations with different workloads. The best performing permutations are stored in the model as workload configurations. The workload configurations are characterized by counts of tasks required to service the workloads. Actual task counts are monitored during normal operation and used to forecast changes in actual task counts. The forecasted task counts are compared with the task counts of the workload configurations of the model to select the best match. Allocation of cores is reconfigured to the best match workload configuration.

Autonomous core affinity

A method and apparatus for enabling dynamic selection between compression algorithms to achieve a target data reduction ratio is provided to predict, on a per-extent basis, whether a write operation targeting a piece of data on a particular extent would be preferentially compressed using either compression algorithm A or compression algorithm B. By selectively implementing compression of particular write operations using compression algorithm B, when it is predicted to achieve a greater compression benefits, it is possible to reduce the overall data reduction ratio achieved by the compression engine while economizing the amount of resources expended by the compression engine. The dynamic selection process determines a per-extent compressibility ratio, which is used in connection with a first threshold to evaluate write operations on a per-extent basis. The dynamic selection processes also determines a per-extent size threshold to select between compression algorithms for particular write operations.

Dynamic selection between compression algorithms to achieve a target data reduction ratio

A random read miss slot size selection engine is configured to select between multiple memory slot sizes to optimize slot size allocations for random read miss IO operations. Upon receipt of an IO operation that is a random read miss IO operation, the slot size selection engine obtains a metadata page encompassing multiple entries in addition to an entry associated with the random read miss IO operation. The slot size selection engine performs a metadata temporal analysis to analyze temporal information associated with previous slot allocations identified in the metadata page. The slot size selection engine also performs a metadata spatial analysis to spatially analyze previous slot allocations to neighboring tracks identified in the metadata page. In response to a determination that the metadata page contains a threshold number of recent slot allocations, the spatial analysis is used to determine the slot size to allocate to the random read miss.

Optimal memory slot size selection for random read miss IO operations

In asynchronous remote replication, write IOs are accumulated in capture cycles and sent to a remote storage system in transmit cycles. In order to cause metadata cache hits at the remote storage system, write IO data and associated metadata hints such as logical block addresses being updated are sent in successive cycles. The metadata hints, which are received at the remote storage system before the corresponding write IO data, are used to prefetch metadata associated with the logical block addresses being updated to replicate the write IO.

Metadata pre-paging on asynchronous remote replication to improve remote array performance

Ransomware activity detection and data protection is implemented by a remote R2 storage array on an asynchronous remote data replication facility, on which data from a primary R1 storage array is replicated to the remote storage array. Write operations on storage volumes in a remote data replication group are collected in a capture cycle on the primary storage array, along with IO pattern metadata describing both read and write operations on the storage volumes. At the end of the capture cycle, the update and metadata is transmitted to the remote storage array. The remote storage array receives the update and metadata and temporarily stores the update prior to applying it to its copy of the storage volumes. Ransomware anomaly detection is implemented using the update and metadata, and if ransomware activity is detected, the data on the remote R2 storage array is protected, and the update is not applied.

Growing community of inventors

Westborough, MA, United States of America

Ramesh Doddaiah