Emc IP Holding Company LLC

Emc Corporation

Snowflake Inc.

Jun Luo

Shashwat Srivastav

Sriram Sankaran

Chen Xiao Wang

Qi Zhang

Vishrut Shah

Huapeng Yuan

Andrew Robertson

Peter Musial

Subba R Gaddamadugu

Jie Song

Wei Yin

Benoit Dageville

Eric Robinson

Nithin Mahesh

Brian Smith

Lin Chan

Tianyi Chen

Yiming Kang

Mikhail Danilov

Stephen Glen Graham

Lu Lei

Ao Sun

Dong Wang

Fredrick Allen Crable

David Gillam

Yu Teng

Igor Medvedev

Ronnie Yu Cai

Jialei Wu

Karthik Navaneethakrishnan

Brian D Burck

Liang Mei

Daquan Zuo

As described herein, a N-Gram index may be created and the search may be conducted using the index, which will lead to faster search results. The N-Gram index may also include partial N-Gram components to capture more relevant data. Moreover, as described herein, the search may also take into account recent log data that has not yet been indexed. Techniques for building an index store using log data and efficiently searching the index store and log data to process search requests are described herein.

Metadata search via N-gram index

Metadata search via N-Gram index

A method, system, and computer program product for providing, via a provisioning engine, a scalable set of indexed key-value pairs enabled to store objects in a data storage environment; wherein the data representing the objects is enabled to be spread across arrays in the data storage environment; wherein additional arrays are enabled to be added to the data storage environment and included in the indexed key-value pairs; wherein the data stored across the arrays may be balanced.

Scalable index store

An efficient and scalable method of managing data protection in a geographically distributed storage system is provided using erasure coding. Each zone in the geographically distributed storage system of multiple zones or clusters encodes primary backup chunks replicated from original data chunks in remote zones to create coding chunks. The zone retains one of the coding chunks and distributes the remaining coding chunks to remote zones for future recovery from dual and more complex failures. Upon completion, the primary backup chunks and peer backup chunks can safely be deleted to reduce overhead. Encoding k primary backup chunks to create m coding chunks, and distributing the m coding chunks to different remote zones assures recovery from 1 to m zone/cluster/chunk failures. After 1 to m zone failures the unaffected zones are each able to recover all of the unavailable chunks by retrieving from each other any of the unaffected zone's distributed coding chunks and still available original data chunks, and restoring the content of the unavailable chunks by decoding the combination of the retained coding chunk, distributed coding chunk, and any still available original data chunks.

Data protection management for distributed storage

A computer implemented method, system, and computer program product for enabling strong consistency of unique objects between zones comprising enabling consistency of an unique data object between in zones by switching between consistency techniques to keep the unique data object consistent between the zones; wherein read and write access is enabled to the unique object in the zones.

Zone consistency

A computer-executable method, computer program product and system for managing metadata in a distributed data storage system, wherein the distributed data storage system includes a first node and one or more data storage arrays, the computer-executable method, computer program product and system comprising partitioning management of metadata created in the distributed data storage system into one or more portions of metadata, wherein the first node manages a first portion of the one or more portions of metadata, and storing the metadata using the first node.

Distributed metadata management

A computer program product, system, and method for generating and storing a parity chunk using at least a first chunk and a second chunk; detecting at least one unused segment within the first chunk; generating a new chunk as a third chunk; associating one or more used segments from the first chunk to the third chunk; filling an unused segment within the third chunk with new data; calculating a delta parity between the first chunk and the third chunk; and updating the stored parity chunk using the delta parity.

Efficient garbage collection for distributed storage with forward error correction

A computer-executable method, computer program product, and system for managing backups in a distributed data storage system including a first zone, a second zone, and a third zone, the computer-executable method, computer program product, and system comprising processing, at the third zone, a first portion of data of the first zone and a second portion of data of the second zone to create a combined portion of data, and removing the first portion and second portion from the third zone.

Data backup management on distributed storage systems

A computer-executable method, computer program product, and system for managing metadata within a distributed data storage system, including a compute node in communication with a data storage array, the computer-executable method, computer program product, and system comprising receiving a data I/O from an application executing within the distributed data storage system, and creating a first storage system within the compute node, wherein the first storage system is enabled to manage metadata related to the data I/O, and processing the data I/O using the first storage system.

Scalable distributed storage system infrastructure

An object storage system having object metadata query capability. Clients can pose queries to the object system in a high-level, declarative language. A query is evaluated against stored object metadata, and only objects satisfying the query's predicates are returned. The object storage system can evaluate alternative logical and physical query plans using a cost model. Queries can be posed using familiar REST-based APIs. The object storage system includes a distributed key-value store for secondary indexing of object metadata. A user can define secondary indexes considering the most common object access patterns.

Object metadata query with secondary indexes

A computer implemented method, computer program product, and system for providing, via a storage provisioning engine, a scalable objects store enabled to store objects across multiple heterogeneous file arrays; wherein file arrays are enabled to be actively added to the object store without pausing the file arrays; and wherein data representing the objects enabled to be balanced across the heterogeneous file arrays based.

Scalable object store

A computer implemented method, system, and computer program product for enabling consistency between zones comprising creating a lease agreement between a first zone and at least a second zone; wherein the lease indicates that the first zone is the owner of an object; wherein the lease agreement dictates that a notification is to be sent before a cached object in the first zone is updated if the lease is still valid; wherein the lease indicates the first zone is to send a heartbeat to the second zone within the predetermined period of time if the object has not been changed and the lease is still valid; and setting up heartbeats from the first zone to at least a second zone within a predetermined amount of time; wherein the heartbeat indicates that the lease is still valid and the object has not been changed.

In one aspect, a method includes using a file system layer configured to interact with a plurality of volumes and enabling an application to interact with any of the plurality of volumes using a single file system format provided by the file system layer. At least two of the plurality of volumes have different file system formats.

File system layer

In one aspect, a method includes enabling a REST interface to have access to a volume, receiving a request to allow a native file access to the volume and allowing an application to use a native file interface to have access to the volume while preventing the modifications to the volume through the REST interface if the request is received.

Growing community of inventors

Bellevue, WA, United States of America

Jun Luo