Comm Vault Systems, Inc.

Hedvig, Inc.

Srinivas Lakshman

Avinash Lakshman

Lásaro Camargos

Suhani Gupta

Anand Vishwanath Vastrad

Abhijith Shenoy

Xiangyu Wang

Srividhya Kavaipatti Anantharamakrishnan

Deepak Jain

Lasaro Camargos

A distributed data storage system using erasure coding (EC) provides advantages of EC data storage while retaining high resiliency for EC data storage architectures having fewer data storage nodes than the number of EC data-plus-parity fragments. An illustrative embodiment is a three-node data storage system with EC 4+2. Incoming data is temporarily replicated to ameliorate the effects of certain storage node outages or fatal disk failures, so that read and write operations can continue from/to the storage system. The system is equipped to automatically heal failed EC write attempts in a manner transparent to users and/or applications: when all storage nodes are operational, the distributed data storage system automatically converts the temporarily replicated data to EC storage and reclaims storage space previously used by the temporarily replicated data. Individual hardware failures are healed through migration techniques that reconstruct and re-fragment data blocks according to the governing EC scheme.

Distributed data storage system using erasure coding on storage nodes fewer than data plus parity fragments

A strongly consistent distributed data storage system comprises an enhanced metadata service that is capable of fully recovering all metadata that goes missing when a metadata-carrying disk, disks, and/or partition fail. An illustrative recovery service runs automatically or on demand to bring the metadata node back into full service. Advantages of the recovery service include guaranteed full recovery of all missing metadata, including metadata still residing in commit logs, without impacting strong consistency guarantees of the metadata. The recovery service is network-traffic efficient. In preferred embodiments, the recovery service avoids metadata service downtime at the metadata node, thereby reducing the impact of metadata disk failure on the availability of the system. The disclosed metadata recovery techniques are said to be 'self-healing' as they do not need manual intervention and instead automatically detect failures and automatically recover from the failures in a non-disruptive manner.

Anti-entropy-based metadata recovery in a strongly consistent distributed data storage system

A distributed data storage system comprises features for integration with application orchestrators such as Kubernetes, and includes a proprietary Container Storage Interface (CSI) driver. Features include setting snapshot scheduling and retention policies, and a 'container data mover' that replicates data from a source to a distinct destination distributed data storage system. The migration may be configured one-to-one, one-to-many, unidirectional, and/or bi-directional. Metadata-based snapshots and metadata-based changed block tracking identify payload data to move from source to destination within application orchestrator frameworks at both ends. Payload data is migrated from source to destination using different techniques than those used for migrating metadata, e.g., kernel-to-kernel for copying payload data versus ordinary metadata writes. An illustrative barrier logic ensures that the migration follows a controlled progression of operations.

Migrating data between data storage systems integrated with application orchestrators

Distributed data storage system using erasure coding on storage nodes fewer than data plus parity fragments and healing failed write attempts

Container data mover for migrating data between distributed data storage systems integrated with application orchestrators

A distributed data storage system using erasure coding (EC) provides advantages of EC data storage while retaining high resiliency for EC data storage architectures having fewer data storage nodes than the number of EC data-plus-parity fragments. To ameliorate the effects of certain storage node outages or fatal disk failures, incoming data is temporarily replicated so that read and write operations can continue from/to the storage system. The system automatically heals failed EC write attempts in a manner transparent to users and/or applications: when all storage nodes are operational, the distributed data storage system automatically converts the temporarily replicated data to EC storage and reclaims storage space previously used by the temporarily replicated data. Individual hardware failures are healed through migration techniques that reconstruct and re-fragment data blocks according to the governing EC scheme. An illustrative embodiment is a three-node data storage system using EC 4+2.

Healing failed erasure-coded write attempts in a distributed data storage system configured with fewer storage nodes than data plus parity fragments

An administrator provisions a virtual disk in a remote storage platform and defines policies for that virtual disk. A virtual machine writes to and reads from the storage platform using any storage protocol. Virtual disk data within a failed storage pool is migrated to different storage pools while still respecting the policies of each virtual disk. Snapshot and revert commands are given for a virtual disk at a particular point in time and overhead is minimal. A virtual disk is cloned utilizing snapshot information and no data need be copied. Any number of Zookeeper clusters are executing in a coordinated fashion within the storage platform, thus increasing overall throughput. A timestamp is generated that guarantees a monotonically increasing counter, even upon a crash of a virtual machine. Any virtual disk has a 'hybrid cloud aware' policy in which one replica of the virtual disk is stored in a public cloud.

Cloning a virtual disk in a storage platform

Disk failure recovery for virtual disk with policies

Time stamp generation for virtual disks

Growing community of inventors

San Jose, CA, United States of America

Srinivas Lakshman