Emc IP Holding Company LLC

Dell Products Inc.

Kevin M Tobin

Andrew Chanler

Michael Ferrari

Gabi Benhanokh

Akshay Kumar Srivastava

Gu Huang

Michael J Scharland

Sandeep Chandrashekhara

Jeffrey Wilson

Jerome Cartmell

Stephen M Lathrop

Nicholas W Von Hein

Direct Image Lookup (DIL) metadata is used to perform differential operation for snapshots and linked targets. Each snapshot contains metadata to store 'images' to perform direct lookup of user data for any part of any snapshot in the system. Metadata pages of DIL image data represent sets of tracks of data. Metadata pages of subsequent snapshots for the same sets of tracks are compared, and where there are no changes to a given metadata page for a given set of tracks between subsequent snapshot copies, the differential process is not run on the tracks associated with the metadata page. If a given metadata page associated with a set of tracks has changed between subsequent snapshots, the differential process is used to identify which tracks in the subsequent snapshot contain different data than the corresponding tracks of the previous snapshot. Similar differential processing also is implemented for relinked target devices.

Direct image lookup for determining quick differential for snapshots and linked targets

Memory management processes allocate and recycle pages of replication data pointer (RDP) metadata space in shared memory. When the RDP page currently allocated to an IO thread becomes full, that RDP page is released, and a new RDP page is allocated to the IO thread. The released page eventually becomes fragmented and is added to a list of RDP pages that are ranked based on partial fullness. An IO thread that needs a new RDP page is allocated a mostly empty RDP page from the ranked list, if such a page is available. Otherwise, a new completely empty RDP page is allocated to the IO thread. Use of the ranked lists reduces latency associated with allocation of new RDP pages. Contention between IO threads for RDP metadata space is reduced because each IO thread has exclusive rights to the free RDP metadata space in its currently allocated page.

Allocating system RDP metadata space with IO performance priority

A node mask of a global metadata structure entry representing an MD portion in GM may be updated when an entry of a local MD table representing the MD portion is removed in response to receiving an IO operation, without negatively impacting performance of the IO operation. An update process that is independent of the process executing the IO operation may update the node mask so that performance of the IO operation is not negatively affected. In response to the entry for the MD portion being removed from the local MD table, an entry may be added to a queue. The update process may include accessing the entries in the queue, and, for each entry, updating the node mask (e.g., clearing a bit representing the processing node) and removing the entry from the queue.

IO metadata management for directly connected host

Snapshots may be managed on a data storage system including logical storage unit including data portions. For a first logical storage unit, a first snapshot pointer structure may be provided including entries, each entry corresponding to a physical storage location at which data is stored for a data portion of the first logical storage unit at a particular point in time. A first virtual snapshot lookup table may be provided for a first portion of the first logical storage unit, the first virtual snapshot lookup table including a plurality of entries, each entry corresponding to a respective data portion of the first logical storage unit and including a reference to a respective entry of the first snapshot pointer structure. The virtual lookup table may correspond to multiple snapshots of the first logical storage unit that have a same value for each data portion of the at least first portion.

Snapshot metadata deduplication

Techniques for obtaining metadata may include: receiving, by a director, an I/O operation directed to a target offset of a logical device, wherein the director is located on a board including a local page table used by components on the board; querying the local page table for a global memory address of first metadata for the target offset of the logical device; and responsive to the local page table not having the global memory address of the first metadata for the target offset of the logical device, using at least a first indirection layer to obtain the global memory address of the first metadata. The global memory may be a distributed global memory including memory segments from multiple different boards each including its own local page table. Compare and swap operations may be used to perform atomic operations to ensure synchronized access when updating the distributed global memory.

Distributed shared memory paging

A host system performs I/O processing using metadata for a storage system, where none or some of the metadata is stored on the host system. The host system may be coupled to the global memory of the storage system along a communication path that includes an internal switching fabric of the storage system and does not include a network located externally to the storage system. The host system may exchange communications over the communication path to access indirection layers on the storage system to determine a global memory address of metadata corresponding to an I/O operation. The host system may include a host metadata table of global memory addresses for metadata of logical locations logical devices. The host system may query the host metadata table for a global memory address of metadata corresponding to a logical location specified in an I/O operation.

Host access to storage system metadata

Sector signature patterns with sector signature values and sector signature parity values are embedded with data slices. A new pattern is embedded with a slice each time the slice data is updated. The patterns are selected in order such that every sector signature value and sector signature parity value changes when a new pattern is embedded. A separate metadata record such as a key is maintained to indicate which pattern has been embedded with the slice. A data integrity check is performed by comparing the embedded sector signature parity values with the metadata record and/or performing a pattern-to-key lookup.

Growing community of inventors

Hopedale, MA, United States of America

Kevin M Tobin