Emc IP Holding Company LLC

Dell Products Inc.

International Business Machines Corporation

Kalyan Chakravarthy Gunda

Jagannathdas Rath

Riyazahamad Moulasab Shiraguppi

Bhimsen Bhanjois

Sandeep R Patil

Nitin Madan

Dean Hildebrand

Manoj Premanand Naik

Santi Gopal Mondal

Subhojit Roy

Kalidas Balakrishnan

Srikant Viswanathan

Srikanth Srinivasan

Jiang Cao

Shashank Prakash Khobragade

George Mathew

Carl Evan Jones

Rahul Deo Vishwakarma

Salil Dangi

Rahul M Fiske

Donna Barry Lewis

Ao Sun

Ramprasad N Chinthekindi

Jayanth Kumar Reddy Perneti

Sanjay Vedanthan

Srisailendra Yallapragada

Kedar Sadanand Godbole

Shyam Burkule

Subrahmanyam Josyula

Alok Katiyar

Chetan Risbud

Yi Ye

Arun Vishnu Pk

Assaf Natanzon

Radia Joy Perlman

Carlos Francisco Fuente

Gururaj Reddy Kulkarni

Shah Mohammad Rezaul Islam

Lu Lei

Robert J Curran

Gary Jialei Wu

Senthil Ponnuswamy

Jian Gong

Wolfgang Mueller-Friedt

Jayasekhar Konduru

Supriya Kannery

Rekha Sampath

Chark Wenshuai Yu

Satish Inampudi

Neeraj Bhutani

Willa Lang Yuan

Yurun Wu

Mukti Jain

Vivek Nambiar

Ravi Kumar Vasarla

Sophie Syau Fang Deng

Zongque Xu

Richard Chenyu Ding

Xiaoxiao Mao

Shu Jiang

Lemonie Mengchi Li

Veera Venkata Prasad Jagarlamudi

The technologies described herein are generally directed toward deallocating storage resources based on metadata of a deleted object. In an embodiment, a method can include, based on an indication that a data object stored using storage resources of a data storage system is to be deleted, identifying metadata of a first chunk of storage of the data storage system. The method can further include based on the metadata of the first chunk of storage, selecting a deallocation approach from different deallocation approaches, the deallocation approach usable to designate the first chunk of storage as available for storage of one or more other data objects. Further, the method can include facilitating designation of the first chunk of storage as being available for the storage of the one or more other data objects in accordance with the deallocation approach.

Deallocating storage resources based on metadata of a deleted object

One example method includes identifying a cloud object as a potential candidate for defragmentation, evaluating the cloud object to determine what portion of segments of the cloud object are expired, when the portion of expired segments meets or exceeds a threshold, segregating the expired segments and unexpired segments of the cloud object, creating a first new cloud object that includes only unexpired segments, creating a second new cloud object that includes only expired segments, and deleting the cloud object from storage.

Efficient cleanup/defragmentation mechanism for expired retention locked (compliance and governance) segments in deduped cloud objects

One example method includes receiving, by a telemetry collection manager from a telemetry consumer, brightness information concerning data accessed by the telemetry consumer, using, by the telemetry collection manager, the brightness information to determine a collection frequency for telemetry concerning the data, and transmitting, by the telemetry collection manager, telemetry collection frequency information to a telemetry generator. In this way, the telemetry consumer is able to control an amount and timing of telemetry generated by the telemetry generator.

System and methods to tune telemetry collection using brightness of data

One example method includes receiving, at a cloud storage site, a write request that identifies an object to be written to the cloud storage site, storing the object at the cloud storage site, receiving, at the cloud storage site, a read request that identifies the object, performing, at the cloud storage site, an evaluation of the object to determine if the object is corrupted, and sending a message identifying an outcome of the evaluation. One or more of the operations at the cloud storage site may be performed by an Object Lambda function that has been configured by a dedupe application.

Efficient method for data invulnerability architecture (DIA) and data corruption detection for deduped cloud objects

Resources of a deduplicated storage appliance are monitored to generate a consumption matrix indicating resource utilization. A threshold map associated with a subsystem is accessed. The subsystem is responsible for an operation involving files stored in the appliance. The threshold map specifies resources required by the subsystem for the operation and thresholds below which multi-streaming for the operation is allowed. An assessment is made as to whether the operation should be handled by multi-streaming. The assessment includes determining whether the resources are underutilized or overutilized based on the consumption matrix and the threshold map. When the resources are overutilized, multi-streaming is not triggered. When the resources are underutilized, multi-streaming is triggered. Periodic reassessments are made to determine whether the operation should be handled by multi-streaming.

Dynamic and adaptive multi-streaming in deduplication systems

One example method includes a data protection operation such as an attack prevention or remediation operation. Policies, such as user policies, role policies, and/or container policies, are created. Prior to attaching to a user or a container, checksums are generated. After the policies are attached to the user or container in the cloud, changes to the policies can be detected using the previously generated checksums and comparing to checksums generated from the cloud policies. Corrective actions are taken when the comparison fails.

Method to detect external attacks on cloud data from within application

A system can determine to write a data object to a storage system that implements an append-only write policy to data chunks. The system can divide a data chunk of the data chunks into a number of data segments. The system can divide respective data segments of the number of data segments into respective stripes. The system can write a first portion of data of the data object to first stripes of the respective stripes. The system can, after writing the first portion of data of the data object to the first stripes, and independently of writing a second portion of data of the data object to second stripes of the respective stripes, determine erasure coding values, and write respective erasure coding values of the erasure coding values to respective erasure coding stripes that are separate from the respective stripes, The system can store the data chunk.

Variable sized data chunks with striped erasure coding

A system can determine a first correlation between respective percentages of stored garbage and respective amounts of garbage of a block storage system based on determining the respective amounts of garbage among first blocks of the respective blocks that satisfy respective criterions of the respective percentages of stored garbage. The system can, based on the first correlation, determine a second correlation between an estimated throughput applicable to reclaiming garbage in the block storage system and the respective amounts of garbage of the block storage system. The system can, based on the first correlation and the second correlation and for a specified target reclamation throughput, determine a corresponding first percentage of stored garbage of the respective percentages of stored garbage. The system can perform copy-forward garbage collection on second blocks of the block storage system that satisfy a criterion defined with respect to the first percentage of stored garbage.

Dynamic tuning of garbage threshold to reduce unreclaimable garbage overhead

The technologies described herein are generally directed toward retrieving data from streaming storage. In an embodiment, a method can include receiving an application data request that identifies application data to be retrieved from a sequence of stored data chunks that correspond to a stored stream of data. The method can further include, based on the application data request, estimating a first estimated location of the application data, with the first estimated location including an identified chunk of a sequence of chunks. Further, the method can include, based on the application data request and a characteristic of the identified chunk, retrieving, by the system, a first data block that is estimated to comprise the application data, resulting in a first retrieved data block.

Retrieving data from streaming storage

One example method includes receiving at a dedupe system, from a client, a request that comprises a set of fingerprints, where each fingerprint in the set corresponds to a particular data segment, filtering, at the dedupe system, the set of fingerprints into a set of unique fingerprints and a set of non-unique fingerprints, reading, at the dedupe system, from a container where copies of the non-unique fingerprints are stored, an additional set of non-unique fingerprints, sending, from the dedupe system to the client, a single response that comprises both the set of unique fingerprints and the additional set of non-unique fingerprints, and receiving from the client, at the dedupe system, data segments that respectively correspond to the unique fingerprints in the set of unique fingerprints, but no data segments corresponding to the non-unique fingerprints in the set of non-unique fingerprints are received by the dedupe system from the client.

Efficient method to optimize distributed segment processing mechanism in dedupe systems by leveraging the locality principle

Embodiments for retention locking a deduplicated file stored in cloud storage by defining object metadata for each object of the file, and comprising a lock count and a retention time based on an expiry date of the lock, with each object having segments, the object metadata further having a respective expiry date and lock count for each segment, where at least some segments are shared among two or more files. Also updating the lock count and retention time for all segments of the file being locked; and if the object is not already locked, locking the object using a retention lock defining a retention time and updating the object metadata with a new lock count and the retention time, otherwise incrementing the lock count and updating the retention time for the expiry date if expiry date of a previous lock is older than a current expiry date.

Extending retention lock protection from on-premises to the cloud

One example method includes, at a replication data source, initiating a replication process that includes transmitting a replication stream to a replication destination vault, and data in the replication stream is transmitted by way of a closed airgap between the replication data source and the replication destination vault, switching, by the replication data source, from a transmit mode to a receive mode, receiving, at the replication data source, a first checksum of a file, and the first checksum and file were created at the replication destination vault, receiving, at the replication data source, the file, calculating, at the replication data source, a second checksum of the file, and when the second checksum matches the first checksum, ending the replication process.

Method to efficiently transfer support and system logs from air-gapped vault systems to replication data sources by re-utilizing the existing replication streams

Containers are generated including data containers storing segments of files, segment tree containers storing upper-level segments of segment trees representing the files, and cloud containers storing headers from the data and segment tree containers. A header for a data container includes fingerprints identifying the segments of files. A header for a segment tree container includes fingerprints identifying the upper-level segments. The containers are sent to cloud storage for storage as a cloud unit. The cloud unit is attached to a deduplicated storage appliance by storing at the appliance the segment tree and cloud containers, cloud configuration details, and an index. The cloud configuration details identify the cloud storage having the cloud unit. The index maps the fingerprints to the containers. When a cloud unit is to be detached, the segment tree and cloud containers, and index from the local storage are deleted, but the cloud configuration details are maintained.

Cloud capacity scaling in metadata space constrained deduplication systems

The disclosure provides a system, computer readable medium, and computer-implemented method for managing data access. The method includes obtaining a data access request from a requestor for client data redundantly stored in at least two storage providers that are unknown to the requestor; identifying a service level objective for servicing the data access request; ranking the at least two storage providers based on the service level objective to identify one of the at least two storage providers to service the data access request; and providing the requestor with access to a copy of the client data using the one of the at least two storage providers.

Method to provide SLA based access to cloud data in backup servers with multi cloud storage

A self-healing method of recovering files using a cyber recovery vault for deduplication filesystems. Data to be backed up from a production system to a backup system in a datacenter is also copied to a cyber security vault coupled to the data center through an air gapped connection. A copy process makes a point-in-time (PIT) copy of the portion of data to a namespace in the vault, and retention locks the copied portion in the vault. An automated process detects a missing segment error, such as due to a bad index lookup in the data to be backed up. The self-healing process provides the missing segment from the PIT copy in the cyber security vault. For any missing fingerprints the process adds the data in new containers and the references in the index, providing an indirection that allows for just writing a new container with the missing L0 references and updating the index.

Self-healing recovery of files using a cyber recovery vault for deduplication file systems

Growing community of inventors

Bengaluru, India

Kalyan Chakravarthy Gunda