Dell Products Inc.

Pure Storage, Inc.

Portworx, Inc.

Quest Software Inc.

Ntt Multimedia Communications Laboratories, Inc.

Vinod Jayaraman

Goutham Rao

Murali Bashyam

Ganesh Sangle

Abhijit Dinkar

Dinesh Israni

Ratna Manoj Bolla

Mark Taylor

Michael E Root

Prabir Paul

Lakshmi Narasimhan Sundararajan

Harsh Desai

Carter George

Parthiban Munusamy

Nikhil Subhash Bhupale

Hitoshi Takanashi

Saurabh Deoras

Luis Pablo Pabón

Micah Wilson

Shekhar Pardeshi

Mike Wilson

Murli Bashyam

Venkatanarasimhan Ramakrishnan

An illustrative computing system may include a master node that includes a prioritization module. The master node may be in communication with each of a plurality of containerized application nodes. The prioritization module may be configured to determine node prioritization information indicating a relative prioritization of the containerized application nodes for instantiating a designated containerized application, wherein the determining node prioritization information includes assigning a priority score to a candidate containerized application node, wherein higher priority is given based on a number of volumes stored on the candidate containerized application node that is designated for access by the designated containerized application.

Hyper-convergence with scheduler extensions for software-defined container storage solutions

An illustrative storage system provides distributed storage for use by containerized applications running in a container system. In some implementations, this includes converting data blocks of storage resources to distributed persistent storage such as thin-provisioned volumes for use by the containerized applications. In some implementations, the conversion includes using a device mapper to virtualize the data blocks of the storage resources to create a virtual storage pool backing datastore in which the thin-provisioned volumes are created.

Converting storage resources to distributed persistent storage for containerized applications

A virtual storage volume may be implemented as one or more chunks stored on a set of storage nodes. The virtual storage volume may be dynamically adjusted by adjusting the one or more chunks that make up the virtual storage volume, without taking the virtual storage volume offline. Such dynamic volume adjustment may allow for increasing volume size without moving data unnecessarily. In addition or alternatively, such dynamic volume adjustment may free up storage space in a node with minimum movement of data.

Dynamic volume adjustment

An illustrative distributed storage system detects a request to resize an online volume that provides persistent storage for a containerized application running in a container system, wherein the online volume is backed by a virtual block device created in device namespace using a device mapper virtualization of data blocks of one or more storage devices. The distributed storage system performs, in response to the request, a resize process to resize the online volume. The resizing of the online volume includes resizing the virtual block device. In some examples, a file system implemented on the virtual block device is also resized while the online volume is online. The storage system may manage inflight IOs over the virtual block device and the online volume such that the resize process is performed without error.

Online resize of a volume of a distributed storage system

A single application may rely on data storage provided by a plurality of virtual data storage volumes mounted on one or more compute nodes. A crash-consistent snapshot of the state of the application may be generated. Each of the virtual data storage volumes may be quiesced prior to the capture of a plurality of volume-level snapshots, after which each volume is unquiesced. The crash-consistent snapshot may be used to restore the application to the state at the time the snapshot was captured.

Snapshotting a containerized application

An illustrative method includes determining that a first application instance of an application is terminated, instantiating, in response to the determining that the first application instance of the application is terminated, a second application instance of the application, identifying one or more particular storage volumes based on the first application instance, and attaching the one or more particular storage volumes to the second application instance.

Systems and methods for regenerating a terminated application instance of a containerized application

In some embodiments, storage path routing in a container system includes: determining, within the container system, a storage operation directed to a storage resource; determining one or more conditions associated with a plurality of network paths to the storage resource, where the plurality of network paths includes a first network path through a storage area network and a second network path through a container system communication network; and routing, based on the one or more conditions, the storage operation to the storage resource using the first network path.

Storage path routing in a container system

An illustrative method includes receiving a request to create a storage volume; identifying, based on the request and on a configuration file propagated among a plurality of storage nodes in a cluster, status indicators of the storage nodes; identifying, based on the request, a size of the storage volume; identifying, based on the request, a replication factor representing a number of storage nodes that are to be replicated within a cluster for the storage volume; identifying, based on one or more characteristics associated with the storage volume, a traffic priority for the storage volume, the traffic priority representing a hierarchy that determines and prioritizes which traffic is allocated to available hardware and network resources in a particular order; creating, based on the status indicators, the size, the replication factor, and the traffic priority, the storage volume on one or more of the plurality of storage nodes.

Traffic priority based creation of a storage volume within a cluster of storage nodes

An illustrative method includes receiving a write request to write payload data to a virtual storage volume; transmitting the write request to a plurality of storage nodes each storing a replica of the virtual storage volume; acknowledging the write request only after a quorum of the storage nodes has stored the payload in their respective kernel memory; and flushing the payloads stored in each kernel memory to persistent storage only after a threshold number of outstanding write requests that have been acknowledged, but not yet flushed, has been reached, the flushing configured to optimize performance for synchronous workloads.

Synchronous workload optimization

An illustrative method includes determining whether a virtual storage volume is successfully mounted to a mount path associated with a compute node, the mount path being marked as read-only, marking, if the determining includes determining that the virtual storage volume is successfully mounted to the mount path, the mount path as writable, and maintaining, if the determining includes determining that the virtual storage volume is unsuccessfully mounted to the mount path, the mount path as read-only.

Mount path management for virtual storage volumes in a containerized storage environment

A virtual storage volume that includes storage space on potentially many different physical disks may be implemented as one or more chunks. This allows for dynamic volume storage adjustment. Dynamic volume storage adjustment allows for increasing volume size without moving data unnecessarily. In addition, dynamic volume storage adjustment also allows freeing up storage space in a node with minimum movement of data.

Dynamic volume storage adjustment

Distributed storage systems are implemented with rule based rebalancing mechanisms. Methods includes steps for creating a set of rules for rebalancing data storage space in a storage node cluster, as well as steps for performing a rebalance operation across the storage node cluster using the set of rules. The distributed storage systems include one or more labels for storage pools and storage volumes.

Cluster rebalance using user defined rules

A clustered storage system may include potentially many different nodes. A node may mount a virtual storage volume for the use of a container application at the node. The node may receive a request from a different node and respond by indicating whether the virtual storage volume is in active use. In this way, the clustered storage system may safely but forcibly unmount a virtual storage volume having a failed or hanging mount point so that the volume may be mounted on a different node.

Resolving failed or hanging mount points in a clustered storage solution for containers

An example method includes identifying, based on a request to write data to a volume provided by storage nodes in a storage node cluster, a cluster hierarchy associated with the storage node cluster, the cluster hierarchy identifying storage characteristics of the storage nodes, wherein the cluster hierarchy is based on data characterizing performance and storage capabilities of the storage nodes; based on the request, selecting a storage node for storing the data in the volume, the selecting being based, at least in part, on the storage characteristics identified by the cluster hierarchy and on one or more data distribution parameters associated with the storage volume; and transmitting the data to the storage node.

Cluster hierarchy-based transmission of data to a storage node included in a storage node cluster

A clustered storage system may include potentially many different nodes, each including a storage driver and a scheduler agent. A node may mount a virtual storage volume for the use of a container application at the node. When a master node receives a request to instantiate a new containerized application, a scheduler at the master node may determine prioritization information for the new containerized application. The prioritization information may prioritize one or more nodes at which a virtualized storage volume employed by the application container instance is mounted.

Growing community of inventors

San Francisco, CA, United States of America

Vinod Jayaraman