International Business Machines Corporation

Rubrik, Inc.

Veza Technologies, Inc.

Maohua Lu

Mihail Corneliu Constantinescu

David Darden Chambliss

Joseph S Glider

Pin Zhou

Prasenjit Sarkar

Ajaykrishna Raghavan

Hyo Jun Kim

Rohit Shekhar

Abdullah Gharaibeh

David Allan Pease

Bhushan Pradip Jain

Anurag Sharma

Tarun Thakur

Nimrod Megiddo

James Lee Hafner

Sangeetha Seshadri

Lawrence Yiumchee Chiu

Paul Henri Muench

Aayush Gupta

Neville Carvalho

Hyojun Kim

The technology disclosed herein enables representation of data access authorizations using a privilege graph. In a particular embodiment, a method includes identifying first attributes of a first user. The method further includes traversing nodes of a privilege graph using the first attributes to determine subsequent nodes until one or more nodes representing a first subset of environments of a plurality of data environments is reached. The method also includes authorizing the first user to access the first subset.

Privilege graph-based representation of data access authorizations

The technology disclosed herein enables pushing of access-privilege information from data environments to a graphing service. In a particular embodiment, a method includes registering a data environment to enable the data environment to use Application Programming Interface (API) calls and receiving an API call transmitted from the data environment. The API call provides information about access permissions for the data environment. The method further includes incorporating the information into a privilege graph representing data access authorizations.

Framework for pushing access-privilege information from data environments

Embodiments disclosed herein provide systems, methods, and computer readable media for sub-cluster recovery in a data storage environment having a plurality of storage nodes. In a particular embodiment, the method provides scanning data items in the plurality of nodes. While scanning, the method further provides indexing the data items into an index of a plurality of partition groups. Each partition group includes data items owned by a particular one of the plurality of storage nodes. The method then provides storing the index.

Sub-cluster recovery using a partition group index

The technology disclosed herein enables generation of effective permissions between principals and resources from access policies. In a particular embodiment, a method includes, in an effective permissions service, retrieving one or more access policies that define access permissions between a principal and a resource of the plurality of resources. The method also includes determining an effective permission defining the access of the principal to the resource based on the access policies and defining the effective permission in a canonical format. The method further includes storing the effective permission for reference when the principal attempts to access the resource.

Effective permissions from IAM (identity and access management) policies

The method disclosed is for instantiating a second cluster based on a first cluster. For at least one node of a second plurality of nodes, generating per node data based on mappings between a plurality of partition groups and a first plurality of nodes, the first plurality of nodes corresponding to the first cluster. The method further discloses identifying data items included in the plurality of partition groups based on the mappings between the plurality of partition groups and the first plurality of nodes. The method further discloses each partition group corresponding to a node of the first plurality of nodes and comprising a subset of data items stored in the node. The method further discloses loading the data items included in the plurality of partition groups onto the second plurality of nodes, the second plurality of nodes corresponding to the second cluster.

The method disclosed includes scanning data items stored in the first plurality of nodes of a first cluster. While scanning, creating a partition group index indexing the data items into a plurality of partition groups. Each partition group corresponds to a node of the first plurality of nodes and comprises a subset of data items stored in the node. Storing the index. Instantiating a second cluster, comprising generating per node data, for each node of a second plurality of nodes, based on mappings between the partition groups and the first plurality of nodes. Identifying the data items included in the partition groups according to the partition group index and loading the data items included in the partition groups onto the second plurality of nodes.

Data deduplication for data storage tapes includes intercepting tape control commands for a single data storage tape. The intercepted tape control commands are modified for adding processing logic and parameters for placement of deduplicated file data on the single data storage tape. Deduplication metadata is written to a metadata portion of the single data storage tape. The deduplicated file data is written to a data portion of the single data storage tape based on the placement to increase read throughput for a deduplicated set of individual files and to reduce an average number of per-file gaps on the single data storage tape without re-duplicating deduplicated data for meeting optimization of individual file accesses.

Optimized data placement for individual file accesses on deduplication-enabled sequential storage systems

Embodiments disclosed herein provide systems, methods, and computer readable media for generating a consistent deduplicated snapshot of a distributed database using optimistic deduplication. In a particular embodiment, a method provides, for each node of a plurality of nodes in the distributed database, deduplicating data items stored on the node that are owned by the node and generating a summary that describes a file in which the data items are located. The method further provides identifying from the summaries for each of the nodes whether mistakes occurred during deduplication and, upon identifying one or more mistakes, determining one or more corrections for the one or more mistakes. Also, the method provides generating a consistent deduplicated snapshot for the distributed database comprising the deduplicated data items from each node and the one or more corrections.

Consistent deduplicated snapshot generation for a distributed database using optimistic deduplication

Embodiments relate to host-side cache migration. An aspect is a method that includes determining pre-fetch hints associated with a source cache that is local to a source host machine. The source cache includes pages of cache data for a virtual machine. The pre-fetch hints are sent to a pre-fetch planner to create a pre-fetch plan. The pre-fetch hints are sent based on migration of the virtual machine to a target host machine including a target cache that is local. At the source host machine, a cache migration request is received based on the pre-fetch plan. A first subset of the pages is sent from the source cache through a host-to-host communication channel to the target cache based on the cache migration request. A second subset of the pages is sent from the source cache through a host-storage communication channel to a shared storage to be relayed to the target cache.

Host-side cache migration

Embodiments disclosed herein provide systems, methods, and computer readable media to enhance the organization, storage, and retrieval of data. In a particular embodiment, a method provides storing a plurality of incremental data versions in a data repository and maintaining first time information for data items in each of the incremental data versions. The method further provides identifying a first data version of the plurality of incremental data versions and writing data in the data repository to create a second data version based the first data version. The method then provides second time information associated with the creation of the second data and creation of the first data version.

Method and apparatus for the generation, organization, storage and retrieval of time stamped blocks of data

Data deduplication for data storage tapes includes intercepting tape control commands for a single data storage tape. The intercepted tape control commands are modified by adding processing logic and parameters for placement of deduplicated file data on the single data storage tape. Deduplication metadata is written to a metadata portion of the single data storage tape. The deduplicated file data is written to a data portion of the single data storage tape based on the placement to increase read throughput for a deduplicated set of individual files and to reduce an average number of per-file gaps on the single data storage tape without re-duplicating deduplicated data for meeting optimization of individual file accesses.

Detecting data duplication includes maintaining a fingerprint directory including one or more entries. Each entry includes a data fingerprint and a data location for a data chunk. A shadow list including a record of fingerprint values not contained in the fingerprint directory is maintained. Each entry is associated with a seen-count attribute, which is an indication of how often a data fingerprint has been seen in arriving data chunks to be written in a storage system, and distinguishes multiply-seen entries for data fingerprints present in at least two data chunks from once-seen entries for data fingerprints present in no more than a single data chunk. Each entry retrieved from the shadow list relates to twice-seen fingerprints.

Growing community of inventors

San Jose, CA, United States of America

Maohua Lu