Emc IP Holding Company LLC

Zvi Schneider

Amitai Alkalay

Assaf Natanzon

Anton Kucherov

Lior Kamran

Xiangping Chen

Felix Shvaiger

Ying Hu

Boris Glimcher

Uri Shabi

Kirill Shoikhet

Tal Ben-Moshe

David Krakov

Gilad Braunschvig

David Meiri

Vladimir Shveidel

Alex Soukhman

Eldad Zinger

Maor Rahamim

Kobi Luz

Niko Farhi

Marcelo Taube

Svetlana Kronrod

Yuval Harduf

Oran Asher Baruch

Aric Hadav

Hillel Costeff

Udi Shemer

Boaz Binia

William F Moore

Inna Resnik

Dani Shemesh

A method for data defragmentation is disclosed, including: selecting a file that is stored in a content-addressable storage system, the file including a plurality of blocks, and the storage system including a plurality of deduplication chunks; arranging the plurality of blocks in block groups, each block group including a set of consecutive blocks having a combined size that matches a deduplication chunk size of the storage system; aligning each block group with a different one of the plurality of deduplication chunks of the storage system, such that the blocks in each block group are stored in the storage system in an order that is based on the order in which the blocks are positioned within the file.

Optimizing file system defrag for deduplicated block storage

A storage system in one embodiment comprises a plurality of storage devices and a storage controller. The storage controller is configured to receive information indicating whether target data is one of a non-deduplicatable type of data and a deduplicatable type of data. Responsive to the information indicating that the target data is of the deduplicatable type, the storage controller is configured to utilize a content-based mapping generated based on a content of the target data to identify a corresponding physical address for storing the target data. Responsive to the information indicating that the target data is of the non-deduplicatable type, the storage controller is configured to utilize an address-based mapping generated based on a logical address associated with the target data to identify a corresponding physical address for storing the target data.

Storage system utilizing content-based and address-based mappings for deduplicatable and non-deduplicatable types of data

Example embodiments of the present invention relate and a method and an apparatus for double hashing. The method including receiving a hash signature, including a short hash handle, for a data block. The method then includes determining a bucket with which the hash signature should be associated and associating the hash signature with the bucket.

Using double hashing schema to reduce short hash handle collisions and improve memory allocation in content-addressable storage systems

In an information processing system including a set of data storage devices for storing data blocks arranged in respective columns on each data storage device and rows across the set of data storage devices to form at least one data stripe, and a set of parity storage devices for storing parity blocks computed via one or more parity operations based on the data blocks of the at least one data stripe, at least one of the data storage devices includes a processing device configured to: receive from the information processing system an instruction to perform at least a portion of a parity operation; perform the portion of the parity operation; and send a result of the performed portion of the parity operation to the information processing system, wherein the result is useable by the information processing system for performing another portion of the parity operation.

Efficient computation of parity data in storage system implementing data striping

A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains data pages associated with an input-output request, provides the data pages to a processing device of a given enclosure, and issues a command to the processing device to perform at least one operation based at least in part on the data pages. The processing device of the given enclosure receives the data pages from the storage controller, generates compressed data pages based at least in part on the received data pages, stores one or more of the compressed data pages on the plurality of drives according to the RAID arrangement and returns information associated with the storage of the compressed data pages to the storage controller.

Compression offloading to RAID array storage enclosure

An aspect includes reading a first page and a corresponding second page from a storage device. The first page specifies a metadata page stored in persistent storage and having logical addresses of metadata, and the second page associates logical block addresses (LBAs) with corresponding physical locations for the metadata. An aspect also includes reading data for a RAID stripe according to an associated physical offset in the second page, accessing a stripe counter from the second page, and comparing the stripe counter from the second page to a stripe counter held in memory. Upon determining the stripe counter from the second page is not the same, a third page is loaded, and a physical location of the data is read from the third page that provides a hash value of the data and corresponding physical location. The physical location of the data is accessed, and the second page is updated.

Fast input/output in a content-addressable storage architecture with paged metadata

A method for performing host based read cache for storage area networks (SANs) supporting Non-Volatile Memory Express over Fabrics (NVMeF) with validation. The method includes upon a first read operation from a client system of a storage array, receiving metadata with a read response regarding location of data subject to the read operation. The method also includes performing, by the client system, direct NVMeF reads from a namespace offset from the metadata. The direct NVMeF reads bypass a software stack of the storage array. The method further includes upon receiving a data block at the client system, using a signature from the metadata to perform validation on the data.

Host based read cache for san supporting NVMEF with E2E validation

An apparatus, in one example, comprises a storage system configured to perform one or more deduplication operations on encrypted datasets received for a plurality of tenants and store at least a portion of the encrypted datasets, the datasets having been encrypted for respective ones of the plurality of tenants using a common encryption key. The apparatus further comprises a cryptographic module associated with the storage system, the cryptographic module configured to, in response to a request to access an encrypted dataset stored by the storage system corresponding to a given one of the plurality of tenants, further encrypt the encrypted dataset using a tenant encryption key associated with the given one of the plurality of tenants. The storage system is further configured to send the further encrypted dataset to the given one of the plurality of tenants that requested access.

Multi-tenant deduplication with non-trusted storage system

In one aspect, data replication processes include setting a minimum snap set creation interval, a maximum snap set creation interval, and a recovery time threshold (RTT). An aspect further includes creating snap sets during a synchronous replication process based on the snap set creation intervals and the RTT. The snap set creation further includes monitoring an amount of data changes since the last snap set creation, monitoring throughput statistics between a source and target system, and estimating an amount of time to replicate the data changes to the target system based on the data changes and throughput statistics. Upon determining the time to replicate the data changes reaches the RTT and the time since the last snap set creation passes the minimum interval, the snap set creation includes creating a next snap set, storing the next snap set at the source, and replicating the next snap set at the target.

Synchronous replication in a storage system

A method is disclosed for storing data in a storage system, comprising: receiving a write request at the storage system; detecting whether the write request is associated with one of a deduplication tier or a non-deduplication tier of the storage system; when the write request is associated with the deduplication tier of the storage system, storing data associated with the write request in the first portion of the volume and updating a first mapping structure to map the first LBA to at least one first hash digest that is generated based on the data; and when the write request is associated with the non-deduplication tier of the storage system, storing data associated with the write request in the second portion of the volume and updating the first mapping structure to map the first LBA to a second hash digest that is generated based on a second LBA.

Sub-lun non-deduplicated tier in a CAS storage to reduce mapping information and improve memory efficiency

A storage system comprises a plurality of enclosures and a storage controller. Each enclosure comprises at least one processing device and a plurality of drives configured in accordance with a redundant array of independent disks (RAID) arrangement. The storage controller obtains an input-output request to read a target data page and issues a command to the at least one processing device of a given enclosure. Responsive to receiving the command, the at least one processing device of the given enclosure obtains data pages associated with a given stripe from the plurality of drives of the given enclosure. The at least one processing device rebuilds the given stripe based at least in part on the obtained data pages, obtains the target data page from the rebuilt given stripe and provides the obtained target data page to the storage controller.

Offloading error processing to raid array storage enclosure

In one aspect, performance optimization for data persistency in asynchronous replication setups includes creating at a source site of a data replication system, a snapshot (snapshot N) of input/output (IO) requests as part of a replication cycle, computing a delta of snapshot N and a previously created snapshot (snapshot N&#x2212;1), and transmitting the delta to a target site of the data replication storage system. An aspect further includes storing, at the target site, snapshot N&#x2212;1 during transmission of the delta, and caching the delta to a non-persistent storage device. Upon determining an occurrence of a loss event at the target site, a further aspect includes transmitting, by the target site, a request to retransmit the delta, retransmitting, by the source site, the delta to the target site, caching the delta to the non-persistent storage device until successful transmission of the delta, and committing the transmitted delta to permanent storage.

Performance optimization for data persistency in asynchronous replication setups

An apparatus in one embodiment comprises at least one processing device comprising a processor coupled to a memory. The at least one processing device is configured to obtain a file and to determine a type of the file. A structure of the file may be determined based at least in part on the determined type of the file and at least one location in the file may be identified based at least in part on the determined structure. The file may be divided at the identified at least one location into a plurality of chunks and the plurality of chunks may be provided to a block deduplication module of a storage system where the block deduplication module is configured to perform a deduplication process based at least in part on the plurality of chunks.

Dual layer deduplication for application specific file types in an information processing system

Described embodiments provide systems and methods for operating a storage system wherein an online consistent system checkpoint is generated. The checkpoint contains a point in time image of a system and is used for providing recovery of the system to a known good state. In one embodiment the checkpoint includes volume configuration data, logical volume space, a plurality of layers of metadata, and physical data storage.

Online consistent system checkpoint

An apparatus comprises a storage system having storage devices and an associated storage controller. In conjunction with initiation of a checkpoint, the storage controller sets a checkpoint started flag for the checkpoint, marks user data pages and metadata pages for write operations already entered in a write journal of the storage system as of the setting of the checkpoint started flag as checkpoint pages, and marks user data pages and metadata pages for new write operations entered in the write journal after the setting of the checkpoint started flag as non-checkpoint pages by altering information used to generate signatures for respective ones of the metadata pages. Metadata pages characterizing the same user data pages subject to write operations at different times thereby have different signatures depending on whether or not the checkpoint started flag was set when its corresponding write operation was entered in the write journal.

Growing community of inventors

Tel Aviv, Israel

Zvi Schneider