Emc IP Holding Company LLC

Emc Corporation

Hui Wang

Xiaomei Liu

Sean C Dolan

Marik Marshak

Owen Martin

Adnan Sahin

Amnon Naamad

Alexandr Veprinsky

Dan Aharoni

Malak Alshawabkeh

Sachin More

Ron Arnan

Tao Kai Lam

Alex Veprinsky

Stephen Richard Ives

Ofer Michael

Qun Fan

Hagay Dagan

John F Madden, Jr

Gil Ratsaby

Ahmet Kirac

Roii Raz

Hagit Brit-Artzi

Michael Dudnik

Gabi Hershkovitz

Arieh Don

Jeffrey Wilson

Jaeyoo Jung

James L Davidson

Dana Naamad

John A Adams

Described are techniques for modeling processing performed in a data storage system. Inputs received may include a plurality of workloads each denoting a workload for one of a plurality of storage groups, a plurality of service level objectives each denoting a target level of performance for one of the plurality of storage groups, a plurality of capacities each denoting a storage capacity of one of a plurality of storage tiers, and a plurality of maximum workloads each denoting a maximum workload capability of one of the plurality of storage tiers. Using the inputs, placement of data of the plurality of storage groups on the plurality of storage tiers may be modeled. Output(s) may be generated based on the modeling where the output(s) may include an amount of each of the plurality of storage tiers allocated by modeling to each of the plurality of storage groups.

Data storage system modeling using application service level objectives and specified workload limits for storage tiers

Forecasting workload activity for data stored on a data storage device includes selecting at least one metric for measuring workload activity, providing at least one grouping of portions of the data according to a workload affinity determination provided for each of the portions at a subset of a plurality of time steps, where the workload affinity determination is based on each of the data portions in the group experiencing above-average workload activity during same ones of the subset of the plurality of time steps, the subset corresponding to at least one business cycle for accessing the data, and forecasting workload activity for all of the portions of data in the group based on forecasting workload activity for a subset of the data portions that is less than all of the data portions.

Extent group workload forecasts

Improved techniques for applying performance controls in a data storage system based on overload correlations between units of managed storage objects (UMOs). When a performance indicator (e.g. response time) for a target UMO is outside an acceptable range, a competing UMO that potentially shares at least one potentially shared hardware component with the target UMO is identified. An overload correlation between the target UMO and the competing UMO is generated that indicates an amount of correlation between the performance indicator for the target UMO and a performance indicator for the competing UMO. A performance control is then applied to the throughput of the competing UMO that reduces the throughput of the competing UMO. The amount that the throughput of the competing UMO is reduced is based on the amount of overload correlation between the target UMO and the competing UMO.

Using overload correlations between units of managed storage objects to apply performance controls in a data storage system

Dynamic resource partitioning techniques may include receiving a plurality of performance goals for a plurality of consumers, each of the plurality of performance goals denoting a performance goal for a different one of the plurality of consumers; performing first processing to determine whether one or more of a plurality of subsystems are overloaded and whether there is at least a first consumer of the plurality of consumers violates a corresponding one of the plurality of performance goals specified for said first consumer; and responsive to determining that one or more of the plurality of subsystems are overloaded and that at least a first consumer is not meeting the corresponding one of the plurality of performance goals specified for said first consumer, performing second processing to reduce workload directed to each of the one or more subsystems that are overloaded.

Dynamic resource partitioning

Described are techniques for controlling I/O response time. It is determined whether a first observed I/O response time for a storage group exceeds I/O performance specified by a service level objective. If so, first processing may be performed to decrease I/O performance for the storage group. The first processing may include: for each I/O directed to the storage group that is received within a first subsequent time period, determining a first response time denoting an amount of time taken to service each I/O prior to returning a response for each I/O; determining whether the first response time for each I/O is less than a response time delay threshold; and if the first response time for each I/O is less than the response time delay threshold, applying a response time delay to each I/O prior to returning a response for each I/O.

Controlling I/O response time to meet service levels

Techniques for obtaining consistent read performance are disclosed that may include: receiving measured read I/O (input/output) response times for flash storage devices; and determining, in accordance with a specified allowable variation, whether a first of the measured read I/O response times for a first of the flash storage devices is inconsistent with respect to other ones of the measured read I/O response times. Responsive to determining the first measured read I/O response time is inconsistent first processing may be performed that corrects or alleviates the inconsistency of the first measured read I/O response time. The first processing may include varying the first measured read I/O response time of the first flash storage device by enforcing, for the first flash storage device, a write I/O workload limit a read I/O workload limit and an idle capacity limit. Data portions may be ranked and selected for data movement based on read workload, write workload or idle capacity. The flash storage may include storage devices of the same type or technology, and the same capacity. Response times for RAID groups may also be measured.

System, method and computer readable medium for obtaining consistent read performance for a plurality of flash drives or raid groups using workload and capacity limits

An external storage resource pool associated with federated tiered storage is associated with at least one performance tier based on evaluated performance of the external storage resource pool. Performance of the external pool may be evaluated in terms of service level expectations or service level objectives. Workload pattern analysis or performance probability curves may be used to classify the external storage resource pool. Workload distribution may be determined by a margin analysis algorithm that matches workloads and storage resource pool performance characteristics.

Classifying performance of an external storage resource pool associated with a performance tier in a federated tiered storage system for overload avoidance and auto-tiering

High performance storage resources are proactively deallocated when a storage group exceeds its service level objective. A load score is calculated for each unit of storage allocated from a high performance physical disk group. A minimum load score is determined that is the lowest load score calculated for the storage units allocated from the high performance disk group to the storage group. When the storage group service level objective is exceeded, a demotion threshold is calculated equal to a product of the minimum load score and a demotion range factor. A demotion set is determined made up of storage units allocated from the high performance disk group having load scores less than or equal to the demotion threshold. Host data stored on the demotion set is migrated to storage from a lower performance disk group, and the high performance units of storage in the demotion set are released for re-allocation.

Proactive release of high performance data storage resources when exceeding a service level objective

Identifying data for placement in a storage system having a plurality of storage classes includes subdividing the data into portions, for each of the portions, independently determining at least one score for a particular portion based on a metric corresponding to access of the particular portion, where the at least one score for the particular portion is independent of scores for other ones of the portions, and identifying sub-portions of data for placement in a particular storage class based on the at least one score of a portion of data corresponding to the sub-portions. The at least one score may be based on short term access statistics and long term access statistics. The access statistics may include read misses, writes, and prefetches.

Indentifying data for placement in a storage system

Described are techniques for processing service level objectives. A first service level objective specified for a storage group of devices may include a first value denoting a first target level of performance for I/O operations. A second service level objective specified for a first portion of the storage group may include a second value denoting a second target level of performance for I/O operations directed to the first portion. The second value may denote a higher level of performance than the first value. It may be determined whether there is a violation of any of the first service level objective and the second level objective. Responsive to determining there is the violation of any of the first service level objective and the second level objective, one or more data movements in accordance with the violation may be performed.

Techniques using multiple service level objectives in connection with a storage group

Described are techniques for configuring data storage. Criteria identifying limits is received for a plurality of storage tiers for one or more logical devices. The storage tiers include a first storage tier and one or more other storage tiers where the first storage tier is higher performing than the other storage tiers. First processing is performed to determine a first set of data portions of the logical devices to be stored on physical devices of the first storage tier subject to limits included in said criteria. First processing includes determining scores for data portions of the logical devices. Each score is calculated using one or more metrics including a first metric weighted based on an expected I/O size. The data portions are ranked in accordance with the scores. The first set of data portions stored in the first storage tier is selected based on the score ranking.

Techniques for automated evaluation and moment of data between storage tiers

Determining cumulative workload skew is described. Measurements for one or more logical devices are determined. The set of measurements include, for each of N storage tiers, a workload measurement identifying workload directed to the single tier, and a capacity measurement identifying an amount of data stored in the single tier. N points may be determined using the measurements. Each point corresponds to a different storage tier and has a first coordinate identifying a cumulative percentage of data portions stored in the storage tier and all other tiers having a higher performance ranking than the one storage tier, and a second coordinate denoting an aggregated percentage of total workload directed to the foregoing cumulative percentage of data portions. A curve representing a cumulative workload skew may be determined using these N points and a point of origin.

Techniques for determining workload skew

When migrating data, a first message is received at a target data storage system from a source data storage system. The target data storage system includes a data storage optimizer that performs automated data movement optimizations. The first message requests a reservation of a first amount of storage on a first storage tier for performing a data migration to migrate data from the source to the target data storage system. A first capacity limit of the first storage tier is reduced by the first amount thereby representing the reservation of the first amount of storage for performing the data migration. If the first storage tier does not include an amount of available storage of at least the first amount, processing is performed to increase the amount of available storage of the first storage tier.

Automatic adjustment of capacity usage by data storage optimizer for data migration

Described are techniques for evaluating data movement alternative. A set of criteria including capacity and performance limits is received. First processing is performed to evaluate a plurality of alternatives for use in data movement with respect to a set of logical devices having data stored on a set of physical storage devices. Each of the plurality of alternatives includes a different set of data movement criteria comprising capacity limits and a different set of performance limits. The set of physical storage devices includes at least a first physical device of one of a plurality of storage tiers and a second physical device of another one of the plurality of storage tier. One of the sets of performance limits is selected in accordance with the first processing.

Techniques for automated evaluation and movement of data between storage tiers for thin devices

Described are techniques for determining whether to move data between devices. A first cost metric is determined for a first device and a second device. At least one portion of data is selected for movement from one of the first device and the second device to another of the first device and the second device. A second cost metric is determined for the first device and the second device based on moving said at least one portion of data between said first device and said second device. It is determined using the first cost metric and the second cost metric whether to move the at least one portion of data.

Growing community of inventors

Upton, MA, United States of America

Hui Wang