Netapp, Inc.

Cisco Technology, Inc.

Springpath, Inc.

Other

Atlantis Computing, Inc.

Shravan Gaonkar

Sandip Agarwala

Deepak Raghu Kenchammana-Hosekote

Smit Shah

Krishna Yadappanavar

Faraz Ahmed Shaikh

Praveen Vegulla

Mallikarjunan Mahalingam

Arthur F Lent

Eric P Forgette

Rahul Narayan Iyer

Sagar Shyam Dixit

Abhishek Chaturvedi

Michael Robin Eisler

Mayuresh Vartak

Sazzala Reddy

Kartikeya Iyer

Jeffrey Zabarsky

Garth Richard Goodson

Kiran Nenmeli Srinivasan

Shankar Pasupathy

Bhavesh Mehta

Prashanth Radhakrishnan

Chetan Venkatesh

Jyothir Ramanan

Deepak Kenchammana-hosekote

Vinodh Dorairajan

Ujwala Prabhakar Tulshigiri

Arif Yakub Patel

Scott Dawkins

Deepak Kenchammana

Systems, methods, and computer-readable media are provided for utilizing distributed erasure encoding in a redundant array of independent disks (RAID) system. An example method can include generating a plurality of virtual redundant array of independent disk (vRAID) stripes, each of the plurality of vRAID stripes including a segment having a plurality of data, each of the plurality of data including metadata, the metadata including a checksum of a corresponding data of the plurality of data, distributing the segment of each of the plurality of vRAID stripes over a plurality of virtual nodes, mapping at least one of logical files, volumes, or objects to the plurality of data chunks and the at least one parity chunk of the plurality of vRAID stripes to avoid write-hole issues, and verifying data integrity of the corresponding data of the plurality of data using the checksum of the corresponding data.

Method and apparatus for distributed erasure encoding

In one embodiment, a device performs a plurality of iterations of write operations on portions of a file tree of a file system to a first storage cluster, each write operation flagging the written portion of the file tree with a generation identifier indicative of the associated iteration of the write operations. The device identifies a set of the written portions of the file tree as unsynchronized between the first storage cluster and a second storage cluster by performing a search for changes made to the written portions of the file tree between a generation identifier associated with the first storage cluster and a generation identifier associated with the second storage cluster. The device synchronizes the first and second storage clusters by sending, via a network, the unsynchronized portions of the file tree from the first storage cluster to the second storage cluster.

Efficient change block tracking in a distributed generation-id (GID) based log structured file system (LSFS)

Techniques for filesystem management for cloud object storage are described. In one embodiment, a method includes writing, by a filesystem layer, a plurality of entries to a log structured file tree, including filesystem metadata and filesystem data. The method includes performing a flush operation of the entries from the filesystem layer to one or more objects in a distributed cloud object storage layer. The method includes storing the filesystem metadata and the filesystem data to the one or more objects in the distributed cloud object storage layer. The method further includes storing flush metadata generated during each flush operation, including a flush sequence number associated with each flush operation. Each object of the one or more objects in the distributed cloud object storage layer is identified by a key that identifies the flush sequence number, an object identifier, and a rebirth identifier.

Filesystem management for cloud object storage

Systems and methods provide for optimizing data deduplication. A physical storage node can determine a cost of performing each exclusive action of a set of exclusive actions associated with data deduplication for a predetermined number of epochs to populate an exclusive action cost matrix. The node can select an optimal exclusive action from among the set of exclusive actions based on the cost of performing each exclusive action. In response to a random number being below a threshold value, the node can determine a first cost of performing a randomly selected exclusive action from the set of exclusive actions and update the exclusive action cost matrix with the first cost. In response to the random number exceeding the threshold value, the node can determine a second cost of performing the optimal exclusive action and update the exclusive action cost matrix with the second cost.

Reinforcement learning for optimizing data deduplication

Techniques for filesystem durable write operations to cloud object storage are described. In one embodiment, a method includes receiving at least one of a read operation or a write operation from a filesystem layer. The read operation identifies content stored in a distributed content object storage layer that is to be retrieved and the write operation identifies content that is to be stored to the distributed content object storage layer. The method includes converting the read operation into a get operation or the write operation into a put operation. The method further includes storing an index that maps the read or write operation to the get or put operation and includes an association between the content identified by the filesystem layer to objects in the distributed content object storage layer. The method includes sending the get or put operation to the distributed content object storage layer for the identified content.

Filesystem durable write operations to cloud object storage

Techniques for implementing distributed quality of service controls for data transmissions between clusters in a clustered distributed storage system are described. In one embodiment, a method of distributed quality of service control includes establishing a control channel between a first plurality of nodes associated with a first cluster. The method also includes setting a quality of service (QoS) policy for the first plurality of nodes that establishes a QoS parameter for data transmitted over a shared network from the first cluster to a second cluster. The method further includes communicating messages between the first plurality of nodes over the control channel to signal an allocated portion of the QoS parameter for each node. The method also includes transmitting data by the first plurality of nodes of the first cluster to a second plurality of nodes of the second cluster over the shared network according to the allocated portion.

Distributed quality of service control for inter-cluster data transmission

Aspects of the subject technology relate to a system configured to receive a request for data associated with a key, identify the key in a first pre-filter in a set of cascading pre-filters for the cache memory, wherein the set of cascading pre-filters is arranged in an order, and store, in response to the key being in the first pre-filter, the key in a second pre-filter in the set of cascading pre-filters, wherein the second pre-filter is next in the order after the first pre-filter.

Cascading pre-filter to improve caching efficiency

I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

I/O bandwidth reduction using storage-level common page information

Creating and using highly available transaction logs in a distributed storage system is described, where the storage controller functions of the distributed storage system are separated from that of distributed storage system storage media. In an exemplary embodiment, a storage controller server receives a transaction entry at a first storage pool of the distributed storage system. The storage controller server further looks up a transaction log to store the transaction entry, where the transaction log is associated with a second storage pool. The storage controller server routes the transaction entry to the second storage pool, wherein the second storage pool stores the transaction entry.

Highly available transaction logs for storing multi-tenant data sets on shared hybrid storage pools

A distributed storage system called StorFS that performs distributed data deduplication is described. In an exemplary embodiment, a storage controller server determines if there is duplicative data in a distributed storage system. In this embodiment, the storage controller server determines if an entity is duplicated in the distributed storage system in line with an incoming input/output operation. The storage controller server determines if the entity is duplicated in the distributed storage system by receiving the entity and looking up the entity in a data deduplication table. If the entity exists in the data deduplication table, the storage controller server updates the metadata for the entity to point to the duplicate entity.

Distributed deduplication in a distributed system of hybrid storage and compute nodes

One or more techniques and/or systems are provided for hosting a virtual machine from a snapshot. In particular, a snapshot of a virtual machine hosted on a primary computing device may be created. The virtual machine may be hosted on a secondary computing device using the snapshot, for example, when a failure of the virtual machine on the primary computing device occurs. If a virtual machine type (format) of the snapshot is not supported by the secondary computing device, then the virtual machine within the snapshot may be converted to a virtual machine type supported by the secondary computing device. In this way, the virtual machine may be operable and/or accessible on the secondary computing device despite the failure. Hosting the virtual machine on the secondary computing device provides, among other things, fault tolerance for the virtual machine and/or applications comprised therein.

Virtual machine fault tolerance

A distributed storage system that dispatches an input/output request is described. In an exemplary embodiment, a storage controller client receives the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers. The storage controller client further computes a target virtual node for the input/output request, where the target virtual node is one of the plurality of virtual nodes. Using the computed target virtual node, the storage controller client determines a target physical node that corresponds to the target virtual node, where the target physical node is one of the plurality of physical nodes. In addition, the storage controller client dispatches the input/output request to a target storage controller that corresponds to the target physical node, wherein the target storage controller server is one of the plurality of storage controller servers. In addition, the virtual node includes a set of one or more mirrored copies across different fault domains for ensuring resiliency and high availability.

Systems and methods for implementing an enterprise-class converged compute-network-storage appliance

A distributed multi-layer cache in a distributed storage system is described, where the storage controller functions of the distributed storage system are separated from that of distributed storage system storage media. In an exemplary embodiment, a storage controller server determines if an object is in a cache that stores a plurality of objects. In addition, the distributed storage system includes the cache and a distributed object layer for persistently storing the plurality of objects. The cache further includes a de-duplicated cache layer. The storage controller server accesses the object from the cache if the object is in the cache and accesses the object from the distributed object layer if the object is not in the cache.

Growing community of inventors

Gainesville, FL, United States of America

Shravan Gaonkar