Netapp, Inc.

Ebay, Incorporated

Sandisk Technologies Inc.

Sandisk Enterprise IP LLC

Harihara Kadayam

Hrishikesh Keremane

Vaiapuri Ramasubramaniam

Rishabh Mittal

Yuri Finkelstein

Chaitanya V Patel

Brian Walter O'Krafka

Johann George

Yong Cho

Parag Sarfare

Prachi Deshmukh

Amarnath Jolad

Birzhan Amirov

Leonid Lokshin

Susan M Coatney

Hari Shankar

Manavalan Krishnan

Sameer Deshpande

Sreelatha Sunkara Reddy

Laurent Nicolas Lambert

Niranjan Patre Neelakanta

Santhosh Unnikrishnan

Frederic H Tudor

Evgeny H Firsov

Ajit Kumar Tapse

Rishab Mittal

Thomas B Bolt

Chaitanya Patel

Yong Eun Cho

Disclosed are systems, methods, and non-transitory computer-readable media for an improved database management system that provides database transactions on a non-transactional database. The database management system executes garbage collection on data stored in a database to remove data values written to the database as part of uncommitted transactions. Each uncommitted transaction is associated with a respective transaction identifier that is not included in a list of committed transaction identifiers. The list of committed transaction identifiers lists, in sequential order, transaction identifiers for committed transaction. After removing each data value written to the database as part of an uncommitted transaction, the database management system modifies the list of committed transaction identifiers to include the transaction identifier for the uncommitted transaction.

Transactions on non-transactional database

In an example embodiment, a distributed storage system includes a service tier including a service node to receive a request for a logical object comprising binary data and metadata describing the binary data, and a storage tier including a plurality of storage nodes, wherein one or more of the storage nodes is to store the metadata describing the binary data. The distributed storage system also includes a coordination tier to store mapping information identifying the one or more of the storage nodes storing the metadata. The service node is also to receive the mapping information from the coordination tier, to access the metadata describing the binary data from one of the one or more of the storage nodes based on the mapping information, and to return the metadata describing the binary data in a response to the request.

Distributed storage of metadata for large binary data

The 'free world replication protocol' makes use of client computing resources, wherein the clients are not part of the replicated key-value store, but instead reside in the 'free world' outside of the dedicated resources of the nodes of the replicated key-value store. In the free world replication protocol, only a single &#x201c;write&#x201d; client is authorized to modify the key-value store at any time but any number of clients may be authorized to read data from the key-value store. The write client sends its transactions to multiple nodes in the replicated key-value store. As a result, the latency between the transaction being sent from the client and the transaction being received by the multiple nodes is reduced. A consensus protocol, driven by a master node, is used to periodically ensure consistency, but the data transactions themselves do not make use of a master node.

Free world replication protocol for key-value store

Systems and methods which provide for managing multiple mirror resources in a storage distribution network are provided. In some embodiments, a system provides for both high availability and disaster recovery functionality at different mirroring locations. Other embodiments may provide for multiple high availability and/or multiple disaster recovery mirror resources. These mirror resources are operated in a heterogeneous manner in the sense that each have its own transport, protocol, and the like, but are configured function cooperatively or as a single mirror with respect to mirroring a primary node. Embodiments may provide for the mirroring and resynchronization of mirrored resources in the event of a communication loss with a particular resource without ceasing the mirroring operations to other resources.

Systems and methods for resynchronizing mirroring partners in a storage system

The 'free world replication protocol' makes use of client computing resources, wherein the clients are not part of the replicated key-value store, but instead reside in the 'free world' outside of the dedicated resources of the nodes of the replicated key-value store. In the free world replication protocol, only a single &#x201c;write&#x201d; client is authorized to modify the key-value store at any time but any number of clients may be authorized to read data from the key-value store. The write client sends its transactions to multiple nodes in the replicated key-value store. As a result, the latency between the transaction being sent from the client and the transaction being received by the multiple nodes is reduced by comparison to existing protocols in which each client sends transactions to a single node and that node forwards the transactions. A consensus protocol, driven by a master node, is used to periodically ensure consistency, but the data transactions themselves do not make use of a master node.

In response to receiving a request to perform a transaction with two or more memory operations on one or more tiered data structures, the memory controller: writes a start transaction record to the log stream including a transaction identifier corresponding to the transaction; and performs the two or more memory operations. For a first memory operation associated with a key, the memory controller: writes a new data object in a datastore; assigns, in a key-map, a location of the new data object to the key; maintains an old data object in the datastore whose location was previously assigned to the key; and writes an operation commit record to a log stream upon completion of the first memory operation. In accordance with a determination that the two or more memory operations are complete, the memory controller writes a transaction commit record to the log stream including the transaction identifier.

Methods and systems for efficient non-isolated transactions

A system and method for handling multi-node failures in a disaster recovery cluster is provided. In the event of an error condition, a switchover operation occurs from the failed nodes to one or more surviving nodes. Data stored in non-volatile random access memory is recovered by the surviving nodes to bring storage objects, e.g., disks, aggregates and/or volumes into a consistent state.

System and method for handling multi-node failures in a disaster recovery cluster

A technique efficiently configures a peered cluster storage environment. The configuration technique illustratively includes three phases: a discovery phase, a node setup phase and a cluster setup phase. The discovery phase may be employed to initiate discovery of nodes of a disaster recovery (DR) group through transmission of multicast advertisement packets by the nodes over interconnects, including a Fiber Channel (FC) fabric, to each other node of the group. In the node setup phase, each node of a cluster assigns its relationships to the nodes discovered and present in the FC fabric; illustratively, the assigned relationships include high availability (HA) partner, DR primary partner and DR auxiliary partner. In the cluster setup phase, the discovered nodes of the FC fabric are organized as the peered cluster storage environment (DR group) configured to service data in a highly reliable and available manner.

Configuration of peered cluster storage environment organized as disaster recovery group

A memory controller receives request(s) to perform a plurality of memory operations on a datastore. The memory controller batches, into a set of leaf node operations, memory operations of the plurality of memory operations that are associated with keys in a range of keys, where a leaf node in a tiered data structure stored in the datastore is assigned the range of keys. The memory controller determines whether a condition is satisfied, and, in accordance with a determination that the condition is satisfied: locates the leaf node in the datastore; stores a copy of the leaf node in a cache; modifies the cached copy of the leaf node according to the set of leaf node operations so as to obtain one or more modified leaf nodes; and writes the one or more modified leaf nodes to one or more locations in the datastore with one or more write operations.

Methods and systems for insert optimization of tiered data structures

Described herein are a system and method for remote mirroring/copying data and metadata sets from a local node to a remote node that reduces the number of metadata sets that are mirrored. In some embodiments, the local node may coalesce metadata sets into metadata chains, each metadata chain comprising a grouping of two or more metadata sets. In some instances, a 'representative' metadata set of a metadata chain may be selected for sending to the remote node for storing, wherein the other metadata sets of the metadata chain are not sent to the remote node. In these embodiments, the selected metadata set may represent all the metadata sets in the chain and be the only metadata set in the chain that is transmitted and stored to the remote node. As such, the network congestion between the local and remote nodes may be reduced.

Coalescing metadata for mirroring to a remote storage node in a cluster storage system

One or more techniques and/or systems are provided for interconnect failover between a primary storage controller and a secondary storage controller. The secondary storage controller may be configured as a backup or failover storage controller for the primary storage controller in the event the primary storage controller fails. Data and/or metadata describing the data (e.g., data and/or metadata stored within a write cache) may be mirrored from the primary storage controller to the secondary storage controller over one or more interconnect paths. Responsive to identifying a failover trigger for a failed interconnect path, the secondary storage controller is instructed to fence (e.g., block) I/O operations from the failed interconnect path. Streams of data and/or metadata that were affected by the failure may be instructed to transmit such data and/or metadata over one or more non-failed interconnect paths to the secondary storage controller during failover of the failed interconnect path.

Growing community of inventors

Fremont, CA, United States of America

Harihara Kadayam