Emc IP Holding Company LLC

Emc Corporation

Sean C Dolan

Hui Wang

Xiaomei Liu

Marik Marshak

Owen Martin

Amnon Naamad

Adnan Sahin

Alexandr Veprinsky

Malak Alshawabkeh

Alex Veprinsky

Dan Aharoni

Ahmet Kirac

Dana Naamad

Sachin More

Alma Dimnaku

Forecasting workload activity for data stored on a data storage device includes selecting at least one metric for measuring workload activity, providing at least one grouping of portions of the data according to a workload affinity determination provided for each of the portions at a subset of a plurality of time steps, where the workload affinity determination is based on each of the data portions in the group experiencing above-average workload activity during same ones of the subset of the plurality of time steps, the subset corresponding to at least one business cycle for accessing the data, and forecasting workload activity for all of the portions of data in the group based on forecasting workload activity for a subset of the data portions that is less than all of the data portions.

Extent group workload forecasts

Improved techniques for applying performance controls in a data storage system based on overload correlations between units of managed storage objects (UMOs). When a performance indicator (e.g. response time) for a target UMO is outside an acceptable range, a competing UMO that potentially shares at least one potentially shared hardware component with the target UMO is identified. An overload correlation between the target UMO and the competing UMO is generated that indicates an amount of correlation between the performance indicator for the target UMO and a performance indicator for the competing UMO. A performance control is then applied to the throughput of the competing UMO that reduces the throughput of the competing UMO. The amount that the throughput of the competing UMO is reduced is based on the amount of overload correlation between the target UMO and the competing UMO.

Using overload correlations between units of managed storage objects to apply performance controls in a data storage system

Dynamic resource partitioning techniques may include receiving a plurality of performance goals for a plurality of consumers, each of the plurality of performance goals denoting a performance goal for a different one of the plurality of consumers; performing first processing to determine whether one or more of a plurality of subsystems are overloaded and whether there is at least a first consumer of the plurality of consumers violates a corresponding one of the plurality of performance goals specified for said first consumer; and responsive to determining that one or more of the plurality of subsystems are overloaded and that at least a first consumer is not meeting the corresponding one of the plurality of performance goals specified for said first consumer, performing second processing to reduce workload directed to each of the one or more subsystems that are overloaded.

Dynamic resource partitioning

Control over the overall data cache hit rate is obtained by managing partitioning caching responsibility by address space. Data caches determine whether to cache data by hashing the data address. Each data cache is assigned a range of hash values to serve. By choosing hash value ranges that do not overlap, data duplication can be eliminated if desired, or degrees of overlap can be allowed. Control over hit rate maximization of data caches having best hit response times is obtained by maintaining separate dedicated and undedicated partitions within each cache. The dedicated partition is only used for the assigned range of hash values.

Controlling aggregate read hit rate across a hierarchy of cache levels by partitioning responsibility for caching among the cache levels

An external storage resource pool associated with federated tiered storage is associated with at least one performance tier based on evaluated performance of the external storage resource pool. Performance of the external pool may be evaluated in terms of service level expectations or service level objectives. Workload pattern analysis or performance probability curves may be used to classify the external storage resource pool. Workload distribution may be determined by a margin analysis algorithm that matches workloads and storage resource pool performance characteristics.

Classifying performance of an external storage resource pool associated with a performance tier in a federated tiered storage system for overload avoidance and auto-tiering

High performance storage resources are proactively deallocated when a storage group exceeds its service level objective. A load score is calculated for each unit of storage allocated from a high performance physical disk group. A minimum load score is determined that is the lowest load score calculated for the storage units allocated from the high performance disk group to the storage group. When the storage group service level objective is exceeded, a demotion threshold is calculated equal to a product of the minimum load score and a demotion range factor. A demotion set is determined made up of storage units allocated from the high performance disk group having load scores less than or equal to the demotion threshold. Host data stored on the demotion set is migrated to storage from a lower performance disk group, and the high performance units of storage in the demotion set are released for re-allocation.

Proactive release of high performance data storage resources when exceeding a service level objective

Identifying data for placement in a storage system having a plurality of storage classes includes subdividing the data into portions, for each of the portions, independently determining at least one score for a particular portion based on a metric corresponding to access of the particular portion, where the at least one score for the particular portion is independent of scores for other ones of the portions, and identifying sub-portions of data for placement in a particular storage class based on the at least one score of a portion of data corresponding to the sub-portions. The at least one score may be based on short term access statistics and long term access statistics. The access statistics may include read misses, writes, and prefetches.

Indentifying data for placement in a storage system

Described are techniques for processing service level objectives. A first service level objective specified for a storage group of devices may include a first value denoting a first target level of performance for I/O operations. A second service level objective specified for a first portion of the storage group may include a second value denoting a second target level of performance for I/O operations directed to the first portion. The second value may denote a higher level of performance than the first value. It may be determined whether there is a violation of any of the first service level objective and the second level objective. Responsive to determining there is the violation of any of the first service level objective and the second level objective, one or more data movements in accordance with the violation may be performed.

Techniques using multiple service level objectives in connection with a storage group

Described are techniques for configuring data storage. Criteria identifying limits is received for a plurality of storage tiers for one or more logical devices. The storage tiers include a first storage tier and one or more other storage tiers where the first storage tier is higher performing than the other storage tiers. First processing is performed to determine a first set of data portions of the logical devices to be stored on physical devices of the first storage tier subject to limits included in said criteria. First processing includes determining scores for data portions of the logical devices. Each score is calculated using one or more metrics including a first metric weighted based on an expected I/O size. The data portions are ranked in accordance with the scores. The first set of data portions stored in the first storage tier is selected based on the score ranking.

Techniques for automated evaluation and moment of data between storage tiers

When migrating data, a first message is received at a target data storage system from a source data storage system. The target data storage system includes a data storage optimizer that performs automated data movement optimizations. The first message requests a reservation of a first amount of storage on a first storage tier for performing a data migration to migrate data from the source to the target data storage system. A first capacity limit of the first storage tier is reduced by the first amount thereby representing the reservation of the first amount of storage for performing the data migration. If the first storage tier does not include an amount of available storage of at least the first amount, processing is performed to increase the amount of available storage of the first storage tier.

Automatic adjustment of capacity usage by data storage optimizer for data migration

Described are techniques for evaluating data movement alternative. A set of criteria including capacity and performance limits is received. First processing is performed to evaluate a plurality of alternatives for use in data movement with respect to a set of logical devices having data stored on a set of physical storage devices. Each of the plurality of alternatives includes a different set of data movement criteria comprising capacity limits and a different set of performance limits. The set of physical storage devices includes at least a first physical device of one of a plurality of storage tiers and a second physical device of another one of the plurality of storage tier. One of the sets of performance limits is selected in accordance with the first processing.

Techniques for automated evaluation and movement of data between storage tiers for thin devices

Techniques for grouping data portions are disclosed. Each group includes data portions determined to exhibit similar behavior. The techniques may include determining whether an affinity measurement with respect to two groups exceeds an affinity threshold; merging the two groups into a single group responsive to the affinity measurement exceeding the affinity threshold; modeling movement of at least one data portion of the single group between two storage tiers at a particular time of day using predicted workload metrics; and performing the data movement of the at least one data portion between the two storage tiers. Predicted workload metrics may be determined by revising first modeled workload metrics using a bias value, where bias values are associated with different times of day, and the bias value is selected based on the particular time of day that the predicted workload metrics are modeling.

Identifying groups of similar data portions

An efficient linear technique is used to determine allocation of tiered storage resources among data extents based on system performance and SLOs. Efficiency is achieved by first determining a system performance boundary condition via hardware performance modeling under desirable system performance zones. SLOs are then balanced and SLO achievement improved by exchanging workload activities among SG donors and SG receivers while system performance boundary conditions are maintained. Remaining unutilized capacity is the uniformly distributed to further improve SLO achievement.

Allocation method for meeting system performance and application service level objective (SLO)

Described are techniques for performing data storage optimizations. A reserved workload for a first of a plurality of storage tiers is determined. Each of the plurality of storage tiers is characterized by a set of one or more attributes. The first storage tier includes performance characteristics which are any of incomplete, unknown, and unable to be predictively modeled for various workloads. The plurality of storage tiers includes the first tier and a remaining set of additional storage tier(s). Performance is modeled of a first workload distributed among the remaining set of storage tiers. The first workload represents a total workload less the reserved workload. One or more data movements are determined in accordance with the modeling. Each data movement moves a data portion from a first physical device of one of the plurality of storage tiers to a second physical device of another of the plurality of storage tiers.

Adaptive techniques for workload distribution across multiple storage tiers

Described are modeling techniques. In accordance with one or more criteria, a determination may be made as to whether to use a dynamic model or a static model to model performance of components, such as storage devices, of a data storage system. A system may include first and second data storage systems where the first data storage system includes a computer readable medium with first code that performs processing in connection with data storage movement optimizations using one or more models including a dynamic model, and second code that generates and maintains the dynamic model used to model performance of storage devices. The second code may include code for performing first processing to determine device sets each of which does not share back-end resources of the second data storage system with any other device sets, and performing second processing to determine sets of performance curves corresponding to the device sets.

Growing community of inventors

Southborough, MA, United States of America

Sean C Dolan