Emc IP Holding Company LLC

Dell Products Inc.

F5 Networks, Inc.

Oran Asher Baruch

Vladimir Shveidel

Vamsi K Vankamamidi

Ronen Gazit

Lior Kamran

Jenny Derzhavetz

Alexander Shknevsky

Maor Rahamim

Anton Kucherov

Amitai Alkalay

Zvi Schneider

Ajay Karri

Peter Finkelshtein

Ami Sabo

Vadim Krishtal

Yuri Chernyavsky

David Ben-David

A method, computer program product, and computer system for receiving, by a computing device, a plurality of IO requests. A portion of the plurality of IO requests may be aggregated based upon a block size. The portion of the plurality of IO requests may be committed to persistent storage in a batch based upon, at least in part, aggregating the portion of the plurality of IO requests based upon the block size.

System and method for aggregation of write commits to control written block size

Techniques of the present disclosure can include: identifying blocks of storage available for allocation; generating a list denoting an allocation order of storage chunks of the blocks; receiving a write I/O operation that writes data to a first logical address; allocating a storage chunk in accordance with the allocation order of the list, wherein a first block includes the storage chunk and a second storage chunk; storing the first data in the storage chunk of the first block; removing the second storage chunk from the list; and creating a mapping between the first logical address and the first block indicating the second storage chunk is reserved for storing content written to a logical address included in a volume logical address range comprising the first logical address. The allocation order can spread allocation distance between blocks and chunks of the same block to avoid contention during flushes.

Storage allocation techniques for large writes based on reserving one or more additional chunks of a block

A method, computer program product, and computing system for generating a page buffer pool within a data journal of a storage node. A plurality of IO operations may be processed on a storage array using the storage node. A plurality of pages may be persisted in a plurality of page buffers within the page buffer pool based upon, at least in part, the processing of the plurality of IO operations. For each page buffer in the page buffer pool, a hash of the page buffer may be generated, thus defining a page buffer hash. The page buffer hash may be stored in a page descriptor associated with the page buffer.

Storage management system and method

A data storage system non-disruptively transitions from using a ring-based write log to using a pool-based write log for buffering data being written to persistent secondary storage. The system initially creates the pool-based write log based on contents of the ring-based write log, and begins to mirror updates of the ring-based write log to the pool-based write log to establish synchronism. Subsequently, and based on an update flag indicating that an update process for the system has not yet completed, the system continues to mirror updates of the ring-based write log to the pool-based write log to maintain synchronism therebetween. Subsequently, and based on the update flag indicating that the update process for the system has completed, the system commences exclusive use of the pool-based write log and making the ring-based write log available for eventual removal.

Data storage system with non-disruptive upgrade using co-existing ring-based and pool-based write logs

A method, computer program product, and computing system for sensing a failure within a system within a computing device. The system may include a cache memory system and a vaulted memory comprising a random access memory (RAM) having a plurality of independent persistent areas. A primary node and secondary node may be provided. The primary node may occupy a first independent persistent area of the RAM of the vaulted memory. The secondary node may occupy a second independent persistent area of the RAM of the vaulted memory. Data within the vaulted memory may be written to a persistent media using an iterator. The data may include at least one dirty page. Writing data within the vaulted memory to the persistent media may include flushing the at least one dirty page to the persistent media.

Storage system and method using persistent memory

In a log based system, a reclamation list of cache pages containing dirty data can be maintained and used. The cached dirty data pages identified by the reclamation list can be candidates for eviction from the cache. A cached dirty data page on the reclamation list can be persistently logged in a transaction log extension having storage allocated from a non-volatile solid state drive (SSD). The reclamation list can include descriptors of the cached dirty data pages prioritized for eviction in accordance with a least recently used (LRU) policy so that the LRU cached dirty data page can be selected for eviction. When a dirty data page which was evicted from the cache is reloaded into cache, the dirty data page can be added to the reclamation list by adding its page descriptor to the reclamation list. A dirty data page can be removed from the reclamation list once flushed.

Techniques for reclaiming dirty cache pages

A method, computer program product, and computing system for receiving a plurality of physical layer blocks (PLBs). A subset of PLBs may be selected from the plurality of PLBs for combining into a combined PLB based upon, at least in part, a utilization of each PLB of the plurality of PLBs, an average compression per active virtual, and a number of free PLBs generated when combining into the combined PLB. One or more PLBs of the subset of PLBs may be compressed based upon, at least in part, the average compression per active virtual. The one or more PLBs of the subset of PLBs may be combined into the combined PLB.

System and method for adaptive data compression

A method may use memory efficiently to extend cache. A processor receives a request to write data. The size of the data in the write request is compared to a threshold. When the size of the data exceeds the threshold, the data is stored on a solid state device. Page descriptors for the data on the solid state device are stored in a metadata log, and a reference to a first page descriptor of the page descriptors in the metadata log is stored in a first hash table in memory.

Memory efficient approach to extending cache

Techniques for rebuilding data in a data storage system are provided. A method includes: (a) identifying (i) a first set of degraded Ubers that contain no portions reserved for direct writing and (ii) a second set of degraded Ubers that contain at least one portion reserved for direct writing. Direct writing is a process that writes blocks to long-term storage prior to mapping those blocks in a metadata mapping structure. An Uber is a set of adjacent stripes across a respective Redundant Array of Independent Disks (RAID) array of the data storage system, and a degraded Uber is an Uber that includes at least one failed drive within its RAID array. The method further includes (b) initiating a rebuild of the first set of degraded Ubers; and (c) delaying a rebuild of each degraded Uber of the second set until all pending direct writes to blocks of that degraded Uber have been mapped by the metadata mapping structure.

Techniques for performing live rebuild in storage systems that operate a direct write mode

A method, computer program product, and computing system for receiving, at a local node, a request to buffer data on a remote persistent cache memory system of a remote node. A target memory address within the remote persistent cache memory system may be sent from the local node via a remote procedure call (RPC). The data may be sent from the local node to the target memory address within the remote persistent cache memory system via a remote direct memory access (RDMA) command.

System and method for multi-node buffer transfer

Write operation may be persistently recorded in a log using PDESC (page descriptor)-PB (page block) pairs. The PDESC-PB pairs of the log may be flushed. Flushing the log may include: determining a working set of PDESC-PB pairs; partitioning the working set into buckets by mapping each PDESC-PB pair of the working set to a bucket using a function; flushing a portion of the PDESC-PB pairs of a first bucket of the working set; updating, at a point in time, a first BHFS (bucket highest flushed sequence ID) value for the first bucket, wherein the first BHFS denotes a first sequence ID and each sequence ID associated with a PDESC-PB pair of the portion flushed prior to the point in time is less than the first sequence ID; and reclaiming PBs of the portion. As part of recovery processing, BHFSs for the buckets may be used to detect invalid PDESCs.

Techniques for log space management involving storing a plurality of page descriptor (PDESC) page block (PB) pairs in the log

Techniques for providing hiccup-less failback and journal recovery. The techniques include determining a head position of a data log of a primary node and placing a marker at a corresponding position in a data log of a secondary node. In response to an IO request, the head of the primary log is moved to allocate space for writing a data entry, and the secondary node is directed to move a head of the secondary log to allocate space for writing the data entry. In response to a data entry being flushed from the primary node, a tail of the primary log is moved to reclaim allocated space for a previous data entry, and the secondary node is directed to move a tail of the secondary log to reclaim previously allocated space. In response to a tail of the secondary log coinciding with the marker position, content of the logs is synchronized.

Hiccup-less failback and journal recovery in an active-active storage system

A method, computer program product, and computer system for receiving, by a computing device, an IO request on a first node. It may be determined whether a virtual address for the IO request is in a virtual cache. A read to RAID may be issued using the virtual address when the virtual address for the IO request is not in the virtual cache. A return of a cached page associated with the virtual address may be issued when the virtual address for the IO request is in the virtual cache.

System and method for deduplication aware read cache in a log structured storage array

A storage system in one embodiment comprises a plurality of storage devices storing data pages. Each data page has a content-based signature derived from that data page. The content-based signatures are associated with physical locations storing the data pages. In response to receipt of a write input/output (IO) request that includes a data segment that is smaller than a page granularity of the storage devices, a content-based signature associated with the data segment is determined which also corresponds to a target data page stored at one of the physical locations. In response to determining the content-based signature, an inflight write count corresponding to the content-based signature is incremented. In response to a decrement request to decrement a reference count of the physical location corresponding to the content-based signature, a decrement flag corresponding to the content-based signature is set in the data structure and the decrement request is postponed.

Storage system with decrement protection of reference counts

A control module of a clustered storage system in one embodiment is configured to receive a write request for a data block of a page, to retrieve an existing content-based signature for the page, to identify from the existing content-based signature a first one of the data modules currently storing the page, and to compute a content-based signature of the data block. The control module is further configured to send the content-based signature of the data block to the first data module, to receive from the first data module an updated content-based signature for the page computed utilizing the content-based signature of the data block, to identify from the updated content-based signature a second one of the data modules that will store the page as updated to include the data block, and to provide information regarding the data block and the updated content-based signature to the second data module for utilization in storage of the updated page.

Growing community of inventors

Tel Aviv, Israel

Oran Asher Baruch