Emc IP Holding Company LLC

Dell Products Inc.

International Business Machines Corporation

Kalyan Chakravarthy Gunda

Jagannathdas Rath

Riyazahamad Moulasab Shiraguppi

Bhimsen Bhanjois

Sandeep R Patil

Nitin Madan

Santi Gopal Mondal

Dean Hildebrand

Manoj Premanand Naik

Subhojit Roy

Kalidas Balakrishnan

Shashank Prakash Khobragade

Srikant Viswanathan

Srikanth Srinivasan

Jiang Cao

Arun Vishnu Pk

George Mathew

Carl Evan Jones

Rahul Deo Vishwakarma

Salil Dangi

Rahul M Fiske

Donna Barry Lewis

Lu Lei

Ramprasad N Chinthekindi

Ao Sun

Jayanth Kumar Reddy Perneti

Sanjay Vedanthan

Srisailendra Yallapragada

Kedar Sadanand Godbole

Subrahmanyam Josyula

Shyam Burkule

Alok Katiyar

Chetan Risbud

Yi Ye

Assaf Natanzon

Radia Joy Perlman

Carlos Francisco Fuente

Gururaj Reddy Kulkarni

Shah Mohammad Rezaul Islam

Robert J Curran

Gary Jialei Wu

Senthil Ponnuswamy

Jian Gong

Chris E Rowen

Wolfgang Mueller-Friedt

Xiao Le Shang

Jayasekhar Konduru

Supriya Kannery

Chark Wenshuai Yu

Rekha Sampath

Willa Lang Yuan

Neeraj Bhutani

Satish Inampudi

Yurun Wu

Heyang Sun

Sophie Syau Fang Deng

Ravi Kumar Vasarla

Vivek Nambiar

Zongque Xu

Richard Chenyu Ding

Mukti Jain

Xiaoxiao Mao

Shu Jiang

Lemonie Mengchi Li

Veera Venkata Prasad Jagarlamudi

Data migrated from a deduplicated storage appliance are stored as cloud units at cloud storage. Each cloud unit includes containers including data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the segments of files. A header for a segment tree container includes fingerprints identifying the upper-level segments. The cloud units are maintained in different states of accessibility including read-write, read-only, and offline. Upon detecting that local storage of the appliance does not have space to support a cloud unit in a read-write or read-only state, another cloud unit that is in a first state is selected, the first state being the read-write or read-only state. The selected cloud unit is placed in a second state, different from the first state.

Cloud capacity scaling in metadata space constrained deduplication systems

One example method includes, at a replication data source, initiating a replication process that includes transmitting a replication stream to a replication destination vault, and data in the replication stream is transmitted by way of a closed airgap between the replication data source and the replication destination vault, switching, by the replication data source, from a transmit mode to a receive mode, receiving, at the replication data source, a first checksum of a file, and the first checksum and file were created at the replication destination vault, receiving, at the replication data source, the file, calculating, at the replication data source, a second checksum of the file, and when the second checksum matches the first checksum, ending the replication process.

Method to efficiently transfer support and system logs from air-gapped vault systems to replication data sources by re-utilizing the existing replication streams

The technologies described herein are generally directed toward writing multiple data objects to object storage. An example method can include receiving an aggregated data object comprising a set of data objects and metadata corresponding to respective data objects of the set of data objects. The method can further include updating the metadata to comprise the respective storage locations. Further, the method can include updating the metadata to include the respective storage locations. The method can further include allocating the metadata to be stored by metadata processing equipment, resulting in the metadata being stored in metadata storage equipment.

Writing multiple data objects to object storage

One example method includes receiving a request to restore a backed up dataset, checking a tracking catalogue to identify any content of the dataset that is prohibited from being restored, preventing restoration of the content, and restoring all of the dataset, except the content, to a restore target. The prohibited content may be one or more files, or portions of files, and may include malware, confidential information, and personal information.

Just in time removal of corrupted info and files from backups on restore

The technologies described herein are generally directed toward deallocating storage resources based on metadata of a deleted object. In an embodiment, a method can include, based on an indication that a data object stored using storage resources of a data storage system is to be deleted, identifying metadata of a first chunk of storage of the data storage system. The method can further include based on the metadata of the first chunk of storage, selecting a deallocation approach from different deallocation approaches, the deallocation approach usable to designate the first chunk of storage as available for storage of one or more other data objects. Further, the method can include facilitating designation of the first chunk of storage as being available for the storage of the one or more other data objects in accordance with the deallocation approach.

Deallocating storage resources based on metadata of a deleted object

One example method includes identifying a cloud object as a potential candidate for defragmentation, evaluating the cloud object to determine what portion of segments of the cloud object are expired, when the portion of expired segments meets or exceeds a threshold, segregating the expired segments and unexpired segments of the cloud object, creating a first new cloud object that includes only unexpired segments, creating a second new cloud object that includes only expired segments, and deleting the cloud object from storage.

Efficient cleanup/defragmentation mechanism for expired retention locked (compliance and governance) segments in deduped cloud objects

One example method includes receiving, by a telemetry collection manager from a telemetry consumer, brightness information concerning data accessed by the telemetry consumer, using, by the telemetry collection manager, the brightness information to determine a collection frequency for telemetry concerning the data, and transmitting, by the telemetry collection manager, telemetry collection frequency information to a telemetry generator. In this way, the telemetry consumer is able to control an amount and timing of telemetry generated by the telemetry generator.

System and methods to tune telemetry collection using brightness of data

One example method includes receiving, at a cloud storage site, a write request that identifies an object to be written to the cloud storage site, storing the object at the cloud storage site, receiving, at the cloud storage site, a read request that identifies the object, performing, at the cloud storage site, an evaluation of the object to determine if the object is corrupted, and sending a message identifying an outcome of the evaluation. One or more of the operations at the cloud storage site may be performed by an Object Lambda function that has been configured by a dedupe application.

Efficient method for data invulnerability architecture (DIA) and data corruption detection for deduped cloud objects

Resources of a deduplicated storage appliance are monitored to generate a consumption matrix indicating resource utilization. A threshold map associated with a subsystem is accessed. The subsystem is responsible for an operation involving files stored in the appliance. The threshold map specifies resources required by the subsystem for the operation and thresholds below which multi-streaming for the operation is allowed. An assessment is made as to whether the operation should be handled by multi-streaming. The assessment includes determining whether the resources are underutilized or overutilized based on the consumption matrix and the threshold map. When the resources are overutilized, multi-streaming is not triggered. When the resources are underutilized, multi-streaming is triggered. Periodic reassessments are made to determine whether the operation should be handled by multi-streaming.

Dynamic and adaptive multi-streaming in deduplication systems

One example method includes a data protection operation such as an attack prevention or remediation operation. Policies, such as user policies, role policies, and/or container policies, are created. Prior to attaching to a user or a container, checksums are generated. After the policies are attached to the user or container in the cloud, changes to the policies can be detected using the previously generated checksums and comparing to checksums generated from the cloud policies. Corrective actions are taken when the comparison fails.

Method to detect external attacks on cloud data from within application

A system can determine to write a data object to a storage system that implements an append-only write policy to data chunks. The system can divide a data chunk of the data chunks into a number of data segments. The system can divide respective data segments of the number of data segments into respective stripes. The system can write a first portion of data of the data object to first stripes of the respective stripes. The system can, after writing the first portion of data of the data object to the first stripes, and independently of writing a second portion of data of the data object to second stripes of the respective stripes, determine erasure coding values, and write respective erasure coding values of the erasure coding values to respective erasure coding stripes that are separate from the respective stripes, The system can store the data chunk.

Variable sized data chunks with striped erasure coding

A system can determine a first correlation between respective percentages of stored garbage and respective amounts of garbage of a block storage system based on determining the respective amounts of garbage among first blocks of the respective blocks that satisfy respective criterions of the respective percentages of stored garbage. The system can, based on the first correlation, determine a second correlation between an estimated throughput applicable to reclaiming garbage in the block storage system and the respective amounts of garbage of the block storage system. The system can, based on the first correlation and the second correlation and for a specified target reclamation throughput, determine a corresponding first percentage of stored garbage of the respective percentages of stored garbage. The system can perform copy-forward garbage collection on second blocks of the block storage system that satisfy a criterion defined with respect to the first percentage of stored garbage.

Dynamic tuning of garbage threshold to reduce unreclaimable garbage overhead

The technologies described herein are generally directed toward retrieving data from streaming storage. In an embodiment, a method can include receiving an application data request that identifies application data to be retrieved from a sequence of stored data chunks that correspond to a stored stream of data. The method can further include, based on the application data request, estimating a first estimated location of the application data, with the first estimated location including an identified chunk of a sequence of chunks. Further, the method can include, based on the application data request and a characteristic of the identified chunk, retrieving, by the system, a first data block that is estimated to comprise the application data, resulting in a first retrieved data block.

Retrieving data from streaming storage

One example method includes receiving at a dedupe system, from a client, a request that comprises a set of fingerprints, where each fingerprint in the set corresponds to a particular data segment, filtering, at the dedupe system, the set of fingerprints into a set of unique fingerprints and a set of non-unique fingerprints, reading, at the dedupe system, from a container where copies of the non-unique fingerprints are stored, an additional set of non-unique fingerprints, sending, from the dedupe system to the client, a single response that comprises both the set of unique fingerprints and the additional set of non-unique fingerprints, and receiving from the client, at the dedupe system, data segments that respectively correspond to the unique fingerprints in the set of unique fingerprints, but no data segments corresponding to the non-unique fingerprints in the set of non-unique fingerprints are received by the dedupe system from the client.

Efficient method to optimize distributed segment processing mechanism in dedupe systems by leveraging the locality principle

Embodiments for retention locking a deduplicated file stored in cloud storage by defining object metadata for each object of the file, and comprising a lock count and a retention time based on an expiry date of the lock, with each object having segments, the object metadata further having a respective expiry date and lock count for each segment, where at least some segments are shared among two or more files. Also updating the lock count and retention time for all segments of the file being locked; and if the object is not already locked, locking the object using a retention lock defining a retention time and updating the object metadata with a new lock count and the retention time, otherwise incrementing the lock count and updating the retention time for the expiry date if expiry date of a previous lock is older than a current expiry date.

Extending retention lock protection from on-premises to the cloud

Growing community of inventors

Bengaluru, India

Kalyan Chakravarthy Gunda