Emc IP Holding Company LLC

Emc Corporation

Xiaomei Liu

Hui Wang

Sean C Dolan

Marik Marshak

Owen Martin

Adnan Sahin

Alexandr Veprinsky

Malak Alshawabkeh

Amnon Naamad

Sachin More

Dan Aharoni

Alex Veprinsky

Ahmet Kirac

Arieh Don

Jeremy O'Hare

Dana Naamad

Forecasting workload activity for data stored on a data storage device includes selecting at least one metric for measuring workload activity, providing at least one grouping of portions of the data according to a workload affinity determination provided for each of the portions at a subset of a plurality of time steps, where the workload affinity determination is based on each of the data portions in the group experiencing above-average workload activity during same ones of the subset of the plurality of time steps, the subset corresponding to at least one business cycle for accessing the data, and forecasting workload activity for all of the portions of data in the group based on forecasting workload activity for a subset of the data portions that is less than all of the data portions.

Extent group workload forecasts

Improved techniques for applying performance controls in a data storage system based on overload correlations between units of managed storage objects (UMOs). When a performance indicator (e.g. response time) for a target UMO is outside an acceptable range, a competing UMO that potentially shares at least one potentially shared hardware component with the target UMO is identified. An overload correlation between the target UMO and the competing UMO is generated that indicates an amount of correlation between the performance indicator for the target UMO and a performance indicator for the competing UMO. A performance control is then applied to the throughput of the competing UMO that reduces the throughput of the competing UMO. The amount that the throughput of the competing UMO is reduced is based on the amount of overload correlation between the target UMO and the competing UMO.

Using overload correlations between units of managed storage objects to apply performance controls in a data storage system

Techniques are described for performing data storage optimization. A first I/O workload for a first data portion of a first snapshot of a first logical device is tracked. First processing is performed by a data storage optimizer to determine a set of one or more data movement optimizations. The first processing uses the first I/O workload for the first snapshot. The set of one or more data movement optimizations include a first data movement that is any of a promotion to move data included in the first data portion from a first storage tier to a higher performance storage tier and a demotion to move data included in the first data portion from the first storage tier to a lower performance storage tier. The first data movement is performed.

Optimizing performance of snapshots based on service level objectives

Techniques for obtaining consistent read performance are disclosed that may include: receiving measured read I/O (input/output) response times for flash storage devices; and determining, in accordance with a specified allowable variation, whether a first of the measured read I/O response times for a first of the flash storage devices is inconsistent with respect to other ones of the measured read I/O response times. Responsive to determining the first measured read I/O response time is inconsistent first processing may be performed that corrects or alleviates the inconsistency of the first measured read I/O response time. The first processing may include varying the first measured read I/O response time of the first flash storage device by enforcing, for the first flash storage device, a write I/O workload limit a read I/O workload limit and an idle capacity limit. Data portions may be ranked and selected for data movement based on read workload, write workload or idle capacity. The flash storage may include storage devices of the same type or technology, and the same capacity. Response times for RAID groups may also be measured.

System, method and computer readable medium for obtaining consistent read performance for a plurality of flash drives or raid groups using workload and capacity limits

An external storage resource pool associated with federated tiered storage is associated with at least one performance tier based on evaluated performance of the external storage resource pool. Performance of the external pool may be evaluated in terms of service level expectations or service level objectives. Workload pattern analysis or performance probability curves may be used to classify the external storage resource pool. Workload distribution may be determined by a margin analysis algorithm that matches workloads and storage resource pool performance characteristics.

Classifying performance of an external storage resource pool associated with a performance tier in a federated tiered storage system for overload avoidance and auto-tiering

High performance storage resources are proactively deallocated when a storage group exceeds its service level objective. A load score is calculated for each unit of storage allocated from a high performance physical disk group. A minimum load score is determined that is the lowest load score calculated for the storage units allocated from the high performance disk group to the storage group. When the storage group service level objective is exceeded, a demotion threshold is calculated equal to a product of the minimum load score and a demotion range factor. A demotion set is determined made up of storage units allocated from the high performance disk group having load scores less than or equal to the demotion threshold. Host data stored on the demotion set is migrated to storage from a lower performance disk group, and the high performance units of storage in the demotion set are released for re-allocation.

Proactive release of high performance data storage resources when exceeding a service level objective

Identifying data for placement in a storage system having a plurality of storage classes includes subdividing the data into portions, for each of the portions, independently determining at least one score for a particular portion based on a metric corresponding to access of the particular portion, where the at least one score for the particular portion is independent of scores for other ones of the portions, and identifying sub-portions of data for placement in a particular storage class based on the at least one score of a portion of data corresponding to the sub-portions. The at least one score may be based on short term access statistics and long term access statistics. The access statistics may include read misses, writes, and prefetches.

Indentifying data for placement in a storage system

Described are techniques for processing service level objectives. A first service level objective specified for a storage group of devices may include a first value denoting a first target level of performance for I/O operations. A second service level objective specified for a first portion of the storage group may include a second value denoting a second target level of performance for I/O operations directed to the first portion. The second value may denote a higher level of performance than the first value. It may be determined whether there is a violation of any of the first service level objective and the second level objective. Responsive to determining there is the violation of any of the first service level objective and the second level objective, one or more data movements in accordance with the violation may be performed.

Techniques using multiple service level objectives in connection with a storage group

Described are techniques for configuring data storage. Criteria identifying limits is received for a plurality of storage tiers for one or more logical devices. The storage tiers include a first storage tier and one or more other storage tiers where the first storage tier is higher performing than the other storage tiers. First processing is performed to determine a first set of data portions of the logical devices to be stored on physical devices of the first storage tier subject to limits included in said criteria. First processing includes determining scores for data portions of the logical devices. Each score is calculated using one or more metrics including a first metric weighted based on an expected I/O size. The data portions are ranked in accordance with the scores. The first set of data portions stored in the first storage tier is selected based on the score ranking.

Techniques for automated evaluation and moment of data between storage tiers

When migrating data, a first message is received at a target data storage system from a source data storage system. The target data storage system includes a data storage optimizer that performs automated data movement optimizations. The first message requests a reservation of a first amount of storage on a first storage tier for performing a data migration to migrate data from the source to the target data storage system. A first capacity limit of the first storage tier is reduced by the first amount thereby representing the reservation of the first amount of storage for performing the data migration. If the first storage tier does not include an amount of available storage of at least the first amount, processing is performed to increase the amount of available storage of the first storage tier.

Automatic adjustment of capacity usage by data storage optimizer for data migration

Described are techniques for evaluating data movement alternative. A set of criteria including capacity and performance limits is received. First processing is performed to evaluate a plurality of alternatives for use in data movement with respect to a set of logical devices having data stored on a set of physical storage devices. Each of the plurality of alternatives includes a different set of data movement criteria comprising capacity limits and a different set of performance limits. The set of physical storage devices includes at least a first physical device of one of a plurality of storage tiers and a second physical device of another one of the plurality of storage tier. One of the sets of performance limits is selected in accordance with the first processing.

Techniques for automated evaluation and movement of data between storage tiers for thin devices

Described are techniques for determining whether to move data between devices. A first cost metric is determined for a first device and a second device. At least one portion of data is selected for movement from one of the first device and the second device to another of the first device and the second device. A second cost metric is determined for the first device and the second device based on moving said at least one portion of data between said first device and said second device. It is determined using the first cost metric and the second cost metric whether to move the at least one portion of data.

Techniques for modeling disk performance

An efficient linear technique is used to determine allocation of tiered storage resources among data extents based on system performance and SLOs. Efficiency is achieved by first determining a system performance boundary condition via hardware performance modeling under desirable system performance zones. SLOs are then balanced and SLO achievement improved by exchanging workload activities among SG donors and SG receivers while system performance boundary conditions are maintained. Remaining unutilized capacity is the uniformly distributed to further improve SLO achievement.

Allocation method for meeting system performance and application service level objective (SLO)

Described are techniques for performing data storage optimizations. A reserved workload for a first of a plurality of storage tiers is determined. Each of the plurality of storage tiers is characterized by a set of one or more attributes. The first storage tier includes performance characteristics which are any of incomplete, unknown, and unable to be predictively modeled for various workloads. The plurality of storage tiers includes the first tier and a remaining set of additional storage tier(s). Performance is modeled of a first workload distributed among the remaining set of storage tiers. The first workload represents a total workload less the reserved workload. One or more data movements are determined in accordance with the modeling. Each data movement moves a data portion from a first physical device of one of the plurality of storage tiers to a second physical device of another of the plurality of storage tiers.

Adaptive techniques for workload distribution across multiple storage tiers

Described are modeling techniques. In accordance with one or more criteria, a determination may be made as to whether to use a dynamic model or a static model to model performance of components, such as storage devices, of a data storage system. A system may include first and second data storage systems where the first data storage system includes a computer readable medium with first code that performs processing in connection with data storage movement optimizations using one or more models including a dynamic model, and second code that generates and maintains the dynamic model used to model performance of storage devices. The second code may include code for performing first processing to determine device sets each of which does not share back-end resources of the second data storage system with any other device sets, and performing second processing to determine sets of performance curves corresponding to the device sets.

Growing community of inventors

Natick, MA, United States of America

Xiaomei Liu