Pure Storage, Inc.

Srinivas Chellappa

Ivan Jibaja

Prashant Jaikumar

Stefan Dorsett

Gaurav Jain

Curtis Pullen

Neil A Vachharajani

Igor Ostrovsky

Neil Buda Vachharajani

Ian Juch

Creating a containerized data analytics pipeline, including: creating a data analytics pipeline having a component that is deployed within a container that executes in a container cluster that is external to shared storage accessible by components of the data analytics pipeline; and responsive to detecting that a component of the data analytics pipeline has failed, deploying another instance of the component in a failover container that is configured to support one or more components of the data analytics pipeline.

Creating a containerized data analytics pipeline

Processing data through a storage system in a data pipeline including receiving, by the storage system, a dataset from a collector on a data producer, wherein the dataset is disaggregated from metadata for the dataset by the collector; storing the dataset on the storage system; receiving, by the storage system from a data indexer, a request for data from the dataset, wherein the request for the data comprises the metadata gathered by the collector on the data producer; servicing, by the storage system, the request for the data by locating the data using the metadata gathered by the collector on the data producer and received in the request for the data; and receiving, from the data indexer, indexed data indexed using the metadata gathered by the collector on the data producer.

Processing data through a storage system in a data pipeline

Providing for high availability in a data analytics pipeline without replicas, including: creating a data analytics pipeline, wherein each component of the data analytics pipeline is deployed within a container; creating a failover container; detecting that a component within the data analytics pipeline has failed; and responsive to detecting that the component within the data analytics pipeline has failed, deploying the component within the data analytics pipeline that has failed in the failover container.

Creating a highly available data analytics pipeline without replicas

Managing a data analytics pipeline

A method of encryption key management in a storage system having a plurality of nodes and more than one key manager, performed by the storage system, is provided. The method includes setting, in a first atomic operation to a distributed store of the plurality of nodes, a version identifier to a new value, and writing shards of a key encryption key, to node-specific memory of the plurality of nodes. The method includes committing the shards of the key encryption key by updating, in a second atomic operation, a set of version identifiers in the distributed store including a current version identifier, responsive to finding no change to the new value of the version identifier.

Distributed, lock-free 2-phase commit of secret shares using multiple stateless controllers

Providing for high availability in a data analytics pipeline without replicas

Growing community of inventors

Sunnyvale, CA, United States of America

Srinivas Chellappa