Emc IP Holding Company LLC

Emc Corporation

Dell Products Inc.

Joris Johannes Wils

William C Davenport

Jack Justin Stiffler

Shari Vietry

Nushafreen Dara Palsetia

Michael James Klemm

Peter Puhov

Socheavy D Heng

Michal Marko

Kumari Bijayalaxmi Nanda

Raju C Bopardikar

Michael J Homberg

Vikram Prabhakar

Beth K Miaoulis

An aspect of space accounting includes providing a counter for each of a plurality of key/value pairs for storage system operations. The key uniquely identifies a logical storage unit on which operations associated with the key are performed, the value identifies an amount of data usage with respect to the operations performed on the logical storage unit, and the counter tracks a type of the operations. An aspect also includes tracking, via corresponding counters, updates to a read/write (RW) tablet associated with a Cuckoo tree. The tracking includes updating a filter to include instances of key/value pairs and the RW tablet in which the pairs are stored. An aspect further includes tracking a plurality of read-only (RO) tablets associated with the Cuckoo tree. The tracking includes updating the filter to include instances of key/value pairs and one or more of the RO tablets in which the pairs are stored.

Space accounting for data storage usage

A method, computer program product, and computing system for processing a snapshot structure of a storage system, wherein the snapshot structure includes one or more temporal indicators and defines a plurality of elements including one or more snapshots, wherein the one or more temporal indicators are incremented to indicate the point at which a data entry was initially added to the snapshot structure. A snapshot matrix is generated based, at least in part, upon the snapshot structure, wherein the snapshot matrix defines the storage space used by one or more eclipsed elements and associated eclipsing elements.

Storage management system and method

A method, computer program product, and computing system for processing a snapshot tree structure of a storage system, wherein the snapshot tree structure defines a plurality of filter elements including one or more snapshots. A snapshot matrix is generated based, at least in part, upon the snapshot tree structure, wherein the snapshot matrix defines the amount of data storage consumed by one or more eclipsed elements and the corresponding eclipsing elements.

A method, computer program product, and computer system for staging writes into a log in chronological order, wherein each write may have a log record of a plurality of log records describing data of the write. The log record may be organized into a bucket of a plurality of buckets associated with a range of a plurality of ranges within a backing store, wherein each bucket of the plurality of buckets may include two keys respectively. The log record of the plurality of log records may be flushed from the bucket of the plurality of buckets to the backing store at a location and in an order determined based upon, at least in part, the two keys included with the bucket.

System and method for balance localization and priority of pages to flush in a sequential log

An aspect of implementing a Cuckoo tree includes providing a Cuckoo filter table for performing lookups in a Cuckoo tree. The table tracks fingerprints of keys and tablets in which the keys reside. An aspect also includes providing a stash configured to manage duplicate keys in the tree. The stash includes: a key hash table that stores full keys corresponding bucket index references that point to a reverse time ordered list of values corresponding to the full keys; and a value store having buckets with slots for storing the list. An aspect further includes setting a duplicate threshold value in the tree that provides a limit on a number of fingerprints that can exist in a bucket. During a store operation on a key, upon determining the number of existing entries with same fingerprint in target bucket would exceed the threshold, another bucket is selected for the key.

Cuckoo tree with duplicate key support

Techniques for storing block-based metadata in a storage system are provided. A method includes (a) receiving a request to write a block of data to a logical address, (b) assigning a physical address where the block of data is to be stored, (c) creating a metadata entry for the block of data, the metadata entry including the physical address, (d) storing a key-value pair within a sorted key-value structure (SKVS) of an ordered plurality of SKVSes, each SKVS having a unique identifier, the key-value pair including the logical address as key and the metadata entry as value; (e) placing an entry within a Cuckoo filter based on the key, the entry including the identifier of the SKVS in which the key-value pair was stored, and (f) subsequently accessing the data by: (1) locating the metadata entry with reference to the Cuckoo filter and (2) locating the data with reference to the physical address in the metadata entry.

Storing metadata in a cuckoo tree

Techniques for providing a cache-based mechanism for snapshot management and creation in a data storage system. The techniques include creating, in a cache memory, a snapshot of a data volume such that the same data is shared between the data volume and the snapshot, and modifying, in the cache memory, one or more data pages among a plurality of data pages of the data volume. The techniques further include maintaining, on a list in the cache memory, copies of the data pages of the snapshot previously shared with the data volume prior to their modification, and maintaining, in a table in the cache memory, a copy record that contains information describing the relationship between the data volume and the snapshot. The techniques still further include flushing, in log order, the data pages of the snapshot and the copy record of the data volume and the snapshot to a storage device.

Systems and methods of managing and creating snapshots in a cache-based storage system

A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet. A given storage element is cloned by making a copy of the given storage element and assigning a storage element address of a new internal identifier to the given source storage element and to the virtual copy of the given source storage element, wherein each storage element is assigned a generation number starting at an initial value.

Storage element cloning in presence of data storage pre-mapper with multiple simultaneous instances of volume address using virtual copies

A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet.

Contention and metadata write amplification reduction in log structured data storage mapping

A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet. Space accounting accounts for usage of the storage devices by a given write operation of a given data offset as one of a new write operation and an overwrite operation.

Space accounting in presence of data storage pre-mapper

A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet. A given storage element is cloned by making a copy of the given storage element and assigning a new internal identifier to the given storage element and the copy.

Storage element cloning in presence of data storage pre-mapper

A method may include generating a chain of snapshot point-in-time copies (PITCs) for a virtual volume, the chain comprising one or more reverse PITCs and one active PITC, such that: (i) read operations to a snapshot pointing to a reverse PITC traverse a portion of the one or more reverse PITCs in a direction of the chain from the reverse PITC pointed to by the snapshot to the active PITC in order to return data responsive to the read operations to the snapshot; and (ii) read operations to the virtual volume traverse only the active PITC in order to return data responsive to the read operations to the virtual volume.

Systems and methods for reverse point-in-time copy management in a storage system

A method and apparatus for partitioning network data services among multiple subscribers uses information stored in several tables to define a plurality of independent networks where each independent network comprises a set of access ports and a unique set of destination addresses and is logically separate from all other independent networks so that no information can pass from one independent network to another independent network. When a subscriber makes a request to use an independent network, a subscriber key that is based on the subscriber identity and the network access port is used to determine whether the subscriber has permission to use the independent network. If the subscriber has permission to use the independent network, information retrieved from the tables is used to determine whether the source subscriber or a destination subscriber has control over the resource. If a destination subscriber has control over the resource, using information in the request to identify a destination subscriber and a response port; and if the source subscriber has control over the resource, using the identity to determine whether the source subscriber can access the resource.

Integrated procedure for partitioning network data services among multiple subscribers

Each time data, in the form of data blocks protected by code checks, must be reformatted, the original data is broken into new data blocks and a new code check is calculated from, and combined with, each new data block, but the new data blocks and new code checks are both reconstituted versions of the original data blocks and the original code checks. Consequently, the data is never left without protection. In one embodiment, an ingress encoder recomputes an ingress code check from an original data block and its associated header. An egress encoder computes an egress code check from the egress header for an outgoing data block reformatted from the original data block and the ingress code check. The outgoing information is then assembled from the egress header, the outgoing data block and the newly computed egress code check.

Growing community of inventors

Acton, MA, United States of America

Joris Johannes Wils