Emc IP Holding Company LLC

Dell Products Inc.

Emc Holding Company, LLC

Vladimir Shveidel

Lior Kamran

Ronen Gazit

Uri Shabi

Amitai Alkalay

Bar David

Geng Han

Dror Zalstein

Anton Kucherov

Vamsi K Vankamamidi

Jenny Derzhavetz

Alex Soukhman

Oran Asher Baruch

Jibing Dong

Yousheng Liu

Vladimir Kleiner

Michael Litvak

Philip C Love

Ami Sabo

David Meiri

Nimrod Shani

Leron Fliess

Roy Koren

Xiangping Chen

Changyu Feng

Zvi Schneider

Socheavy D Heng

Shaul Dar

Kirill Shoikhet

Svetlana Kronrod

Maor Rahamim

Gajanan S Natu

Alexander Shknevsky

David Bernard

Itai Shavit

Adi Katzengold

Philippe Armangau

Xinlei Xu

Shaoqin Gong

Jianbin Kang

Christopher Alan Seibel

Felix Shvaiger

Ying Hu

Steven A Morley

Tal Ben-Moshe

Leonid Ravich

Samuel L Mullis, Ii

Shari Vietry

David Krakov

Haiyun Bao

Jason E Raff

Alexei Kabishcer

Christopher A Seibel

Nir Bar Joseph

Christopher J Jones

Liang Huang

Benny Kirenberg

Seman Shen

Philip Love

David Bar

Yair Hershkovitz

Dennis Rusakov

Dafna Levi-Yadgar

Peleg Vigodny

Daniel Krauthgamer

Jonathan Volij

Gang Han

Aleksey Kabishcher

A shared portion of processor cores in a data storage system are allocated for sharing between a storage system application and a containerized service also executing in the data storage system. A non-shared portion of the processor cores are allocated for exclusive use by the storage system application. The storage system application preferentially uses processor cores in the shared portion of the processor cores to execute background tasks, and preferentially uses processor cores in the non-shared portion to perform host I/O request processing. The storage system application may from time to time voluntarily yield one or more of the processor cores in the shared portion of the processor cores for execution of the containerized service.

Asymmetric central processing unit (CPU) sharing with a containerized service hosted in a data storage system

Regulating maintenance background (BG) task activity using instrumented waiting points (WPs) can include: collecting feedback metrics for the WPs, where each WP is instrumented to collect a set of feedback metrics including an average waiting time and a first value denoting a portion of resource requests received at the WP which are from maintenance BG tasks; determining, in accordance with criteria and collected feedback metrics, whether the criteria is true for at least a first WP associated with a set of feedback metrics, wherein the criteria includes determining whether the first WP has an associated average waiting time of the set which exceeds a waiting time threshold, and whether the first WP has a share of maintenance BG requests exceeding a minimum; and responsive to determining that the criteria is true for at least the first WP, performing processing to reduce a current activity level of said maintenance BG tasks.

Maintenance background task regulation using feedback from instrumented waiting points

Multiple co-resident flush working sets are created in a data storage system. Each flush working set indicates host data that was received by the data storage system from at least one host computing device during a period of time corresponding to that flush working set. Host data indicated by the flush working sets is to be flushed from a storage processor of the data storage system to non-volatile data storage devices of the data storage system, and indications of host data within the flush working sets are organized according to logical block addresses of the indicated host data. Multiple independently executing flusher threads flush the host data indicated by at least one of the flush working sets from the storage processor to the non-volatile data storage devices of the data storage system.

Creating and using multiple flush working sets in a data storage system

Techniques for implementing metadata streams that provide increased performance and resiliency. The techniques include writing raw delta updates of metadata pages to a metadata delta log, and generating a metadata stream for a metadata page containing bulk updates. The techniques include passing the metadata stream to a tiered storage array, and storing the metadata page in a physical block on a metadata page tier. The techniques include, in response to storing the metadata page in the physical block, updating, in local memory, parity data for a RAID stripe including the metadata page. The techniques include, once the physical block is filled with metadata pages, storing the parity data from the local memory to a parity page of the stripe. The techniques include, in response to a storage drive failure when the physical block is partially filled with metadata pages, rebuilding the stripe data using the parity data in local memory.

Smooth metadata pages stream to raid blocks in log structured metadata based storage cluster

Techniques for using data logs with bonded page descriptors (PDs) in storage systems. The techniques include, in response to a write request from a storage client, performing, in a log structured data log, a single object allocation for a bonded page descriptor (PD)-page buffer (PB) pair, storing and persisting, in a PB of the bonded PD-PB pair, user data specified by the write request, and storing and persisting, in a PD object of the bonded PD-PB pair, metadata related to the user data. The techniques include, once the user data and related MD are stored and persisted in the bonded PD-PB pair, sending an acknowledgment message to the storage client, and de-staging, flushing, or transferring, in the background, the user data and/or related MD from the bonded PD-PB pair to a storage object. By performing a single object allocation, processing costs related to performing the object allocation can be reduced.

Data log with bonded descriptors

A searchable metadata change log is maintained in a persistent memory of a data storage system, and stores metadata changes to be aggregated and applied to data storage system metadata located in non-volatile data storage. New metadata changes are added to the metadata change log by storing the new metadata changes into the persistent memory, storing structural modifications to the metadata change log needed to add the new metadata changes to the metadata change log into a transaction entry located in the persistent memory, and processing the resulting transaction entry to perform the structural modifications to the metadata change log that were stored in the transaction entry.

Managing changes to metadata in a data storage system

In at least one embodiment, processing can include receiving at a first node a read I/O operation that requests to read content stored at a logical address LA owned by a second node. Responsive to the second node receiving a request regarding LA from the first node, the second node can perform address resolution processing using mapping information for LA that includes: an address PAof a page Pof storage of a first PLB (physical large block) where Cis stored, a value Vof a generation identifier GenIDof Pand the first PLB, and a hash value HVfor C. A reply, as returned from the second node to the first node, can include information including PAand hints including Vand HV. The first node can perform processing to determine, using the information, whether Cstored at PAwas overwritten during a vulnerability window.

Techniques for providing data consistency

A technique for managing reference-count decrements (decrefs) includes defining an aggregation domain that includes multiple tablets created at respective points in time, each of the tablets containing multiple sorted buckets arranged in a respective sequence of bucket identifiers that is repeated across all of the tablets, the sorted buckets containing decrefs for respective sets of metadata pages. The technique further includes assigning a decref job to a specified set of bucket identifiers and destaging, by the decref job, multiple decrefs from the tablets to persistent metadata pages, beginning with decrefs in buckets having the specified set of bucket identifiers and proceeding until a target number of decrefs has been destaged.

Managing decref processing from tablets containing sorted buckets

In at least one embodiment, destaging a write of a user data (UD) log may include: performing deduplication that determines content written by the write to a logical address LA is a duplicate of existing content; and establishing mapping information of a chain of metadata (MD) pages mapping LA to a physical address PA storing the existing content, wherein the chain includes a MD leaf page and a VLB (virtual layer block) page. An update of a MD log can update an indirect pointer (IDP) field of a MD leaf entry of the MID leaf page to reference a VLB entry of the VLB page where the VLB entry further includes PA. A second update of the MD log can increment a reference count of the VLB entry. The second update can be recorded blindly in the MD log without reading or accessing the reference count during destaging the write.

Efficient deduplication in a metadata delta log architecture

A method, computer program product, and computing system for receiving a metadata page for storage within a log-structured metadata store. An entry may be generated within a translation table page of a translation table within the log-structured metadata store, wherein each entry of a plurality of entries within the translation table page is associated with a separate metadata page. One or more metadata page deltas for the metadata page may be recorded in the log-structured metadata store. The one or more metadata page deltas for the metadata page may be flushed to a persistent memory storage. In response to flushing the one or more metadata page deltas for the metadata page, the entry within the translation table page that is associated with the metadata page may be updated without accessing any other entry within the translation table page associated with other metadata pages.

System and method for managing metadata access in a log-structured system

In at least one embodiment, processing can include: maintaining a storage object identifier hash table (SOIDHT) that tracks storage object identifiers (IDs) that have been reassigned among storage objects and that have potential outstanding activity associated with storage objects as prior to performing corresponding reassignments; recording, in a user data (UD) log, a UD log entry corresponding to an operation directed to a storage object assigned a storage object ID, wherein the UD log entry includes a first LSN denoting a point in time when the operation is recorded in the UD log; and flushing the UD log. Responsive to determining that the storage object has a corresponding SOIDHT entry, processing can include determining whether the first LSN of the first UD log entry of the operation is less than a second LSN of the corresponding entry of the SOIDHT.

Storage space accounting techniques allowing storage object identifier reuse and reassignment

In at least one embodiment, processing can include: receiving, at a first node of a system, a cross slice border (CSB) I/O with a target location that crosses a CSB between two slices owned by different nodes; adding an entry for the CSB and CSB I/O to a cross border hash table (CBHT) of the first node; acquiring first locks for logical addresses of the first slice owned by the first node; sending a remote procedure call (RPC) to a second node requesting second locks for second logical addresses of the second slice owned by the second node; receiving an RPC reply in response to the RPC; and responsive to receiving an RPC reply indicating the negative acknowledgement, releasing the first locks and removing the entry from the CBHT of the first node.

Techniques for providing I/O atomicity and preventing deadlock for cross slice border I/Os in a distributed storage system

In at least one embodiment, processing can include: receiving metadata updates for MD pages; recording the MD updates for the MD pages in a MD log; and performing a first destaging phase that writes the MD updates from the MD log to a tablet stored on non-volatile storage. The tablet can include a bucket area and an extension area. The bucket area can include buckets each of which is a predetermined bucket size. The first destaging phase can include: writing a portion of the MD updates from the MD log to the bucket area; and writing a second portion of the MD updates from the MD log to the extension area. Each MD update of the second portion can overflow a corresponding one of the buckets which is included in the bucket area and which is associated with one of the MD pages updated by the MD update.

Techniques using a variable size extension area with two phase destaging

In one embodiment, processing can include: receiving, at a first node, a read I/O to read content C1 of logical address LA, wherein the first node does not own LA; and responsive to determining a hash table of the first node does not include a matching entry for LA, performing processing including: sending a request from the first node to a second node that owns LA; sending, to the first node, a response including an address hint for LA and additional address hints for logical addresses, wherein LA and the additional address hints are included in a logical address subrange associated with a single metadata page used in mapping LA and the logical addresses to physical storage locations that store content of LA and the logical addresses; the first node adding entries to the hash table for the additional address hints; and obtaining C1 using the address hint for LA.

Prefetching by logical address owner in response to peer node mapping request

In at least one embodiment, processing can include receiving, at a first node, a read request directed to a logical address LA owned by a second node. The first node can locally cache content and an address hint corresponding to LA. The first node can issue a request to the second node. The request can include a flag to suppress the second node from returning content stored at the target logical address. The first node can receive a response including an address used to read current content of LA1 from back-end non-volatile storage. The first node can determine whether the address matches the address hint cached on the first node. If the first node determines the address and address hint match, the cached content of LA stored on the first node is valid and can be returned in response to the read as current content stored at LA.

Data caching strategies for storage with ownership of logical address slices

Growing community of inventors

Pardes-Hana, Israel

Vladimir Shveidel