Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Amc IP Holding Company LLC

Nickolay Alexandrovich Dalmatov

Mikhail Danilov

Richard P Ruef

Kurt William Everson

Jian Gao

Michael P Wahl

Kirill Alexsandrovich Bezugly

Assaf Natanzon

Ilya Usvyatsky

Kirill Aleksandrovich Bezugly

Vladimir Shatunov

Alexey Fomin

William Carl Biester

Yuri Zagrebin

Roman Vladimirovich Marchenko

Alexander Valeryevich Romanyukov

Alexander Shraifel

Techniques for handling data with different lifetime characteristics in stream-aware data storage systems. The data storage systems can include a file system that has a log-based architecture design, and can employ one or more solid state drives (SSDs) that provide log-based data storage, which can include a data log divided into a series of storage segments. The techniques can be employed in the data storage systems to control the placement of data in the respective segments of the data log based at least on the lifetime of the data, significantly reducing the processing overhead associated with performing garbage collection functions within the SSDs.

Handling data with different lifetime characteristics in stream-aware data storage equipment

A method and system is used in managing deduplication of data in storage systems. A first digest for a deduplication candidate is received. At least one stream associated with the deduplication candidate is detected. At least one neighboring digest segment of a first loaded digest segment associated with the at least one stream is loaded. Whether the digest is located in the at least one neighboring digest segment is determined. If the digest is not located in the at least one neighboring digest segment, the digest is processed.

Inline deduplication using stream detection

A method for storing data in a system that includes a plurality of storage devices, the method that includes obtaining object usage data from the plurality of storage devices, determining, using the object usage data, object clusters, where at least one object cluster of the object clusters includes at least two objects that are associated based on access patterns, migrate a first object, of the two objects, from a first storage device of the plurality of storage devices to a second storage device of the plurality of storage devices.

Systems and methods for object migration in storage devices

A technique manages data in slices of difference sizes within different storage tiers. The technique involves, based on access activity for first data currently residing within a first slice having a first size, selecting a target set of storage devices within which to store the first data from among multiple sets of storage devices. The technique further involves moving the first data from the first slice having the first size to a second slice having a second size that is different from the first size. The technique further involves, after the first data is moved from the first slice to the second slice, storing the second slice in the target set of storage devices.

Storing data in slices of different sizes within different storage tiers

Allocation of parity data storage extents for a virtual RAID is disclosed. A virtual RAID can be enabled by a virtual storage device pool (VSDP) mapped to a physical storage device pool (PSDP) that can comprise physical extents of physical disks. In an aspect, the physical disks can be co-located, remotely located, or combinations thereof. Mapping of the VSDP can enable allocation of virtual extents in a virtual RAID that can, for example, emulate a RAID4 while still providing parity data storage diversity that can, for example, emulate RAID5, RAID6, etc. Moreover, the disclosed subject matter can support proactive wear leveling, for example, based on historical storage of parity data via an extent, e.g., extents previously used to store parity data. Furthermore, the disclosed subject matter can support active wear leveling.

Parity allocation for a virtual redundant array of independent disks

A deduplication index is generated having multiple entries, each entry storing a digest of a data block that was previously stored in non-volatile data storage together with a pointer to the location in non-volatile storage at which the data block was previously stored. The entries of the disclosed deduplication index are divided into multiple deduplication index segments. A resident subset of the deduplication index segments is stored in memory of the data storage system. A non-resident subset of the deduplication index segments is stored in non-volatile data storage of the data storage system. Data deduplication is performed for each subsequently received data block for which a digest is generated that matches any one of the digests in the entries of the deduplication index segments that are contained in the resident subset of the deduplication index segments.

Segmented index for data deduplication

A method, computer program product, and computer system for identifying, by a computing device, information associated with a relationship between a physical layer block and a virtual logic block for RAID storage. The information associated with the relationship between the physical layer block and the virtual logic block may be written within the RAID storage. The physical layer block within the RAID storage may be rebuilt only when the physical layer block includes the information associated with the relationship between the physical layer block and the virtual logic block.

System and method for faster RAID rebuild via allocation knowledge

A technique for managing SSDs in a data storage system generates an endurance value for each of multiple SSDs and arranges the SSDs in RAID groups based at least in part on the generated endurance values. As a result of such arranging, some RAID groups may include only SSDs with higher endurance values while other RAID groups may include only SSDs with lower endurance values. The data storage system may then run RAID groups with higher endurance values at higher speeds and may run RAID groups with lower endurance values at lower speeds.

Arranging SSD resources based on estimated endurance

A method for provided use in a storage device having a controller, the method comprising: identifying, by the controller, a plurality of logical regions in the storage device; obtaining, by the controller, a respective usage metric for each of the logical regions; updating, by the controller, a translation data structure of the storage device, the translation data structure being updated to map any of the logical regions of the storage device to a respective physical portion of the storage device, the respective physical portion being selected based on the respective usage metric of the logical region, wherein the translation data structure is part of a flash translation layer of the storage device, and the translation data structure is configured to store mapping information between a logical address space of the storage device and a physical address space of the storage device.

Method and apparatus for storage device management

Techniques for data placement may include receiving data portions stored at logical addresses, and storing the data portions on slices of physical storage located in storage tiers. The storage tiers may include different size slices on the different tiers. In one embodiment, slices in the same tier are all the same size. In another embodiment, slices in the same tier may be of different sizes. Slices of data may be demoted and promoted among the storage tiers as the workloads of the slices changes over time. Demotion may include combining slices into a larger slice. Promotion may include partitioning a slice into smaller slices. Additionally, multiple slices of a tier may be combined into a larger slice in the tier. A slice in the tier may be partitioned into multiple smaller slices also located in the tier.

Migation of data portions between different sized slices in the same of different performance tier

A technique manages file data of a file system. The technique involves provisioning a first LUN with slice locations from a heterogeneous storage pool created from a solid state device (SSD) storage tier formed from SSD memory and a hard disk drive (HDD) storage tier formed from HDD memory. The technique further involves provisioning a second LUN with slice locations from the heterogeneous storage pool. The technique further involves, while different LUN level policies are applied to the first LUN and the second LUN, moving file data of a file system from the slice locations of the first LUN to the slice locations of the second LUN. Such a technique enables effective auto-tiering at the file level where active file data moves to a higher performance storage tier and inactive file data moves to a cost effective lower performance storage tier thus optimizing operation.

Managing a file system within multiple LUNS while different LUN level policies are applied to the LUNS

A technique is directed to performing a tuning operation in data storage equipment. The technique involves generating, while the data storage equipment performs input/output (I/O) transactions, an observed I/O statistics profile based on performance of at least some of the I/O transactions. The technique further involves performing a comparison operation that compares the observed I/O statistics profile to an expected I/O statistics profile which is defined by a set of operating settings that controls operation of the data storage equipment. The technique further involves operating the data storage equipment in a normal state when a result of the comparison operation indicates that the observed I/O statistics profile matches the expected I/O statistics profile and in a remedial state which is different from the normal state when the result of the comparison operation indicates that the observed I/O statistics profile does not match the expected I/O statistics profile.

Tuning data storage equipment based on comparing observed I/O statistics with expected I/O statistics which are defined by operating settings that control operation

A technique manages data within solid state device (SSD) storage. The technique involves, in response to writing data to a set of SSD storage components, consuming a set of recurring write quotas for the set of SSD storage components. Each recurring write quota identifies an amount of remaining usefulness for a respective SSD storage component, e.g., periodically allocated budgets for write operations based on measured (or counted) reliability and/or healthiness factors. The technique further involves, as the set of recurring write quotas are consumed, performing a set of quota evaluation operations to evaluate the set of recurring write quotas. The technique further involves, in response to a set of results from the set of quota evaluation operations, performing a set of remedial activities to control access to the data that was written to the set of SSD storage components.

Managing data within solid state drive storage based on recurring write quotas

A technique balances data storage activity within a mapped-RAID environment. The technique involves selecting, by processing circuitry, a source slice of storage from multiple slices of storage of the mapped-RAID environment, the source slice containing particular data to be relocated. The technique further involves selecting, by the processing circuitry, a destination slice of storage from the multiple slices of storage of the mapped-RAID environment. The technique further involves relocating, by the processing circuitry, the particular data from the source slice to the destination slice to balance data storage activity within the mapped-RAID environment. The mapped-RAID environment includes multiple storage devices. Each storage device provides multiple non-overlapping device extents. Each slice of the multiple slices of storage of the mapped-RAID environment is formed of storage stripes extending across device extents provided by a group of storage devices that includes less than all of the storage devices of the mapped-RAID environment.

Growing community of inventors

Saint Petersburg, Russia

Nickolay Alexandrovich Dalmatov