Dell Products Inc.

Emc IP Holding Company LLC

Emc Corporation

Stephen M Lathrop

Jeremy O'Hare

Anoop Raghunathan

Michael J Scharland

Roy E Clark

Kiran Prakash Madnani

Wai Chuen Yim

Kuolin Hua

Kevin M Tobin

Yongmei Xie

David K Aha

Christopher W Appleby

Chaitanya Lohani

Garret Bourke

Roman Gramc

Bingfeng Zhao

Jun Chen

A storage system includes storage resources, front-end devices, and back-end devices that are subdivided into allocation units. Each front-end device has a plurality of tracks, which map to entries in the allocation units. Front-end metadata is used to specify, for a respective track, a respective back-end device, an allocation unit within the respective back-end device, and an entry within the respective allocation unit where the data for the respective track is stored. Back-end metadata is used to specify, for a respective back-end entry, a respective location in the storage resources of the storage system where the data for the respective track is stored and a size of the data on the storage resources. The back-end metadata is organized using shared back-end metadata memory pages, in which each page of metadata is equally shared between two or more of the allocation units.

Direct addressing and efficient storage of variable sized metadata

A task generator runs on a data storage system in which multiple compute nodes allocate portions of local memory to a remotely accessible shared memory. The task generator is responsive to a primary task to be performed on a storage object to generate separate, per-compute node secondary tasks corresponding to the primary task. Each of the separate secondary tasks specifies at least one attribute of metadata associated with the storage object and at least one task logic function. Each of the plurality of compute nodes performs the separate secondary task generated for that compute node by scanning the local portion of the shared memory based on the attribute to identify matching pages of the metadata associated with the storage object and performs the task logic function on the identified matching pages of the metadata associated with the storage object to generate a local result. The task generator combines the local results to perform the primary task.

Scanning pages of shared memory

Techniques for processing data may include: receiving source physical storage allocation units that include valid data and one or more holes of unused physical storage, wherein each source physical storage allocation unit has an associated timestamp denoting a most recent destaging time of any data stored on the source physical storage allocation unit; determining an age for each source physical allocation unit based on the associated timestamp of the source physical allocation unit; for each source physical storage allocation unit, determining one of multiple age buckets for the source physical allocation unit based on the age of the source physical allocation unit; and for a first of the age buckets including two source physical allocation units, performing first processing including: relocating at least some valid data from the two source physical storage allocation units to a target physical storage allocation unit.

Defragmentation techniques

Sector signature patterns with sector signature values and sector signature parity values are embedded with data slices. A new pattern is embedded with a slice each time the slice data is updated. The patterns are selected in order such that every sector signature value and sector signature parity value changes when a new pattern is embedded. A separate metadata record such as a key is maintained to indicate which pattern has been embedded with the slice. A data integrity check is performed by comparing the embedded sector signature parity values with the metadata record and/or performing a pattern-to-key lookup.

Metadata pattern to detect write loss/inconsistencies of optimized-write-once operations

Protection and recovery is provided for an allocation unit of data that is in-flight between a source device and a target device of a storage system during a systemic failure by selecting between corresponding source device and target device metadata based on consistency and reciprocal linkage. If logical volume metadata for the allocation unit of the data in volatile memory is not consistent with logical volume metadata for the allocation unit in non-volatile storage, then the allocation unit is recovered from the source device. If target device metadata for the data in volatile memory is not consistent with target device metadata for the data in non-volatile storage, then the allocation unit is recovered from the source device. If linkage between logical volume metadata for the allocation unit and target device metadata for the data is not reciprocal, then the allocation unit is recovered from the source device. If none of those conditions are satisfied, then the allocation unit is recovered from the target device.

Efficient resilience in a metadata paging array for in-flight user data

The system, devices, and methods disclosed herein relate to data ratio reduction technology adapted to reduce storage costs by weeding out duplicative data write operations. The techniques and systems disclosed achieve deduplication benefits of smaller block, finer granularity, deduplication systems while simultaneously reducing the enormous metadata costs traditionally associated with small block deduplication. According to an exemplary 128 K block deduplication system and method, we disclose deduplication rates nearly equivalent to 4 K deduplication systems.

Large block misaligned deduplication

A technique for logging events in a data storage system involves designating one subset of storage processors of the data storage system as clients that generate log entries and another subset of the storage processors as servers that receive log entries. Only one server is active at a time. The active server receives the generated log entries from the clients and persists the log entries to a centralized log store. Clients assign first timestamps to the log entries based on locally accessible clocks. The active server receives the log entries, including the first timestamps, from the clients and applies second timestamps based on a clock accessible to the server. As the second timestamps are consistent across the different clients, the second timestamps can be applied to correct misalignments in time among the log entries received from the clients.

Centralized logging for a data storage system

An aggregator device provides an iSCSI client with a list of storage areas which are offered by a single aggregated network array. The storage areas on the list are provided by multiple data storage systems and are accessible by the client using data paths which are out-of-band with the aggregator device. The aggregator device further receives an iSCSI login request from the client. The iSCSI login request identifies a particular storage area on the list of storage areas. The aggregator device further generates and sends an iSCSI login response to the client in response to the iSCSI login request. The iSCSI login response includes a network address of a particular data storage system of the multiple data storage systems which provides the particular storage area to enable the client to directly access the particular storage area using a data path which is out-of-band with the aggregator device.

Growing community of inventors

Milford, MA, United States of America

Stephen M Lathrop