Emc IP Holding Company LLC

Dell Products Inc.

Emc Holding Company, LLC

Bar David

Vladimir Shveidel

Dror Zalstein

Ronen Gazit

Uri Shabi

Michael Litvak

Philip C Love

Jenny Derzhavetz

Bar Harel

David Meiri

Xiangping Chen

Amitai Alkalay

Alex Soukhman

Shari Vietry

Jason E Raff

David Bernard

Ami Sabo

Adi Katzengold

Dennis Rusakov

A method, computer program product, and computing system for receiving a plurality of requests to perform an operation of a first operation type on a storage object. A shared write lock associated with the first operation type is provided to each thread requesting to perform an operation of the first operation type. Concurrent requests to perform operations of the first operation type on the storage object are processed by each requesting thread based upon, at least in part, the shared write lock associated with the first operation type provided to each requesting thread.

System and method for sharing write locks for specific operation types

Processing may be performed in accordance with a policy to assign roles of winner and loser between two nodes. The roles may be used in connection with deadlock resolution processing. A deadlock or potential deadlock may be detected between the two nodes performing processing for two transactions In response to detecting the deadlock or potential deadlock, using a current state may be used to determine whether to perform the deadlock resolution processing to resolve the deadlock or potential deadlock. The current state may indicate whether assignment of the winner and loser roles between the two nodes is in progress. Responsive to the current state indicating that processing is not in progress to assign roles of winner and loser between the two nodes, the current state may be used perform deadlock resolution processing to resolve the deadlock or potential deadlock. The current state may denote which node is the current winner.

Balanced winner assignment for deadlock resolution

A technique is directed to processing metadata changes. The technique involves designating a first tablet to ingest metadata changes and a second tablet to destage previously ingested metadata changes, the first tablet being partitioned into a first reserved space and a first regular space that form a first memory pool, and the second tablet being partitioned into a second reserved space and a second regular space that form a second memory pool. The technique further involves, while the first tablet is designated to ingest metadata changes and the second tablet is designated to destage previously ingested metadata changes, ingesting metadata changes into the first tablet and destaging the previously ingested metadata changes from the second tablet. The technique further involves, when the first tablet becomes full, performing a switch operation that designates the second tablet to ingest metadata changes and the first tablet to destage previously ingested metadata changes.

Maintaining smooth metadata log destaging and burst tolerance

A method is used for improving performance in a storage system during deduplication. A processor receives a request to write data. A determination is made that the data can be deduplicated. A non-exclusive lock for a page in storage with a reference count of the data may be obtained, and an instruction to increment the reference count for the data may be stored in a metadata journal. The non-exclusive lock for the page may be released.

Method and system for improving performance during deduplication

A method, computer program product, and computer system for acquiring, by a first node, local locks of the first node associated with a metadata log transaction, wherein the first node acquires the local locks of the first node prior to sending a commit message to a second node. The second node may acquire local locks of the second node associated with the metadata log transaction, wherein the second node acquires the local locks of the second node based upon, at least in part, receiving the commit message from the first node.

System and method for consistent metadata access based on local locks semantics in a multi-node cluster

A method, computer program product, and computer system for receiving, by a computing device, a Write-Same operation from a host for a range of logical block addresses of a destination. Data may be recorded in a buffer to indicate that the Write-Same operation is complete prior to completing the Write-Same operation. An acknowledgment may be sent to the host that the Write-Same operation is complete prior to flushing to a final destination. The Write-Same operation for the logical block addresses of the destination may be performed after sending the acknowledgment to the host that the Write-Same operation is complete.

System and method for near-instant unmapping and write-same in a log-structured storage cluster

In a dual node system, each node can store metadata updates in a volatile memory metadata log. Metadata pages can be used to access data stored on a volume. A node designated as preferred or affined for a volume can be expected to receive I/Os directed to the volume. A preferred node for a volume can record, in its volatile memory metadata log, the metadata changes for the metadata pages used to access data stored on the volume. A non-preferred or non-affined node for the volume can infrequently receive I/Os directed to the volume. A non-preferred node for a volume can record, in its volatile memory metadata log, pointers or references to the metadata changes for the metadata pages used to access data stored on the volume, where the pointers or references map to persisted copies of the meta changes as stored in a persisted metadata log.

Techniques for recording metadata changes

Techniques for efficiently flushing a user data log may postpone or delay establishing chains of metadata pages used as mapping information to map logical addresses to storage locations of content stored at the logical addresses. Processing can include: receiving a write operation that writes data to a logical address; storing an entry for the write operation in the user data log; and flushing the entry from the user data log. Flushing can include storing a metadata log entry in a metadata log, wherein the metadata log entry represents a binding of the logical address to a data block including the data stored at the logical address; and destaging the metadata log entry. Destaging can include updating mapping information used to map the logical address to the data block. The mapping information can include a metadata page in accordance with the metadata log entry.

Techniques for efficient user log flushing with shortcut logical address binding and postponing mapping information updates

In at least one embodiment, processing can include: receiving a request for a transaction of MD (metadata) updates including a first MD update of a first MD update type of a first set and including a second MD update of a second MD update type of a second set; storing, in a first volatile MD log, the first MD update; storing, in a second volatile MD log, the second MD update; storing, in a first non-volatile MD log, the first MD update; and storing, in a second non-volatile MD log, the second MD update, wherein each MD update of the first volatile MD log and the first non-volatile MD log has a corresponding MD update type of the first set, and each MD update of the second volatile MD log and the second non-volatile MD log has a corresponding MD update type of the second set.

Techniques for efficient journal space handling and recovery processing with multiple logs

Processing a read request to read metadata from an entry of a metadata page may include: determining whether the metadata page is cached; responsive to determining the metadata page is cached, obtaining the first metadata from the cached metadata page; responsive to determining the metadata page is not cached, determining whether the requested metadata is in a metadata log of metadata changes stored in a volatile memory; and responsive to determining the metadata is the metadata log of metadata changes stored in the volatile memory, obtaining the requested metadata from the metadata log. Processing a write request that overwrites an existing value of a metadata page with an updated value may include: recording a metadata change in the metadata log that indicates to update the metadata page with the updated value; and performing additional processing during destaging that uses the existing value prior to overwriting it with the updated value.

Techniques for metadata updating and retrieval

In at least one embodiment, processing can include receiving a metadata (MD) update for a MD page, wherein the MD update has an associated type; determining whether the type is a custom MD type; responsive to determining the type is a custom MD type, performing first processing including: determining a custom bucket handle for the MD update based at least in part on the type and a logical index (LI) uniquely identifying the MD page; and applying the MD update to a custom bucket associated with the custom bucket handle; and responsive to determining the type is a custom MD type, performing second processing including: determining a regular bucket handle for the MD update based at least in part on the LI of the MD page; and applying the MD update to a regular bucket associated with the regular bucket handle.

Techniques for performing metadata updates

Techniques for providing dependency resolution for lazy snapshots in a storage cluster with a delta log-based architecture. The techniques include creating a dependency relationship between a parent metadata page and a child metadata page, receiving one or more new delta updates of the parent metadata page, and writing, to a metadata delta log, the new delta updates of the parent metadata page as specialized delta update entries that preserve or maintain the new delta update values and their corresponding original (old) delta update values. By preserving or maintaining both the new delta update values and the corresponding old delta update values of the parent metadata page in the metadata delta log, constraints pertaining to the order of de-staging the delta updates of the parent and child metadata pages can be reduced.

Dependency resolution for lazy snapshots in storage cluster with delta log based architecture

In at least one embodiment, processing can include acquiring a spinlock on a cached copy of a metadata (MD) page includes a field stored in two cache lines; updating a register to include an updated value of the field; determining whether a first portion of the updated value of the register is non-zero, wherein two portions of the updated value of the field as stored in the register correspond to the two cache lines; and responsive to determining that the first portion of the updated value of the register is non-zero, performing processing including: storing the first portion of the updated value of the field from the register in the first cache line; and subsequent to performing storing the first portion, storing the second portion of the updated value of the field as stored in the register in the second cache line.

Techniques for performing metadata updates for cache consistency

In at least one embodiment, processing can include receiving a metadata (MD) structure including MD pages; and performing a MD split operation with respect to a first of the MD pages, wherein said performing the MD split operation includes: generating a first ALI (abstract logical index) representing a new MD page that is unallocated and is a child of the first MD page; and storing an entry in a bucket of an in-memory MD log for the first ALI, wherein the entry denotes a mapping between the first ALI and a corresponding LI (logical index), wherein the entry indicates that the corresponding LI associated with the first ALI is invalid since the first ALI represents a new MD page which is unallocated and not associated with physical storage; and destaging the in-memory MD log, wherein said destaging includes allocating first physical storage for the new MD page.

Resource allocation techniques using a metadata log

A method, computer program product, and computing system for receiving a flush request for a metadata page stored in a storage array of a multi-node storage system. The flush request may be queued on a flush request lock queue on at least one node of the multi-node storage system. One or more flush requests may be processed, via multiple nodes of the multi-node storage system, on the metadata page based upon, at least in part, the flush request lock queue.

Growing community of inventors

Rishon Lezion, Israel

Bar David