Emc IP Holding Company LLC

Tipper Truong

Philip N Shilane

Joseph S Brandt

Nicholas A Noto

Kimberly R Lu

Mariah Arevalo

Abhinav Duggal

One example method includes identifying objects that each include one or more segments to be transferred from a source storage tier to a target storage tier, determining a total amount of data to be transferred, using a tiering controller to create worker nodes operable to transfer the segments to the target storage tier, where a number of worker nodes created is based on the amount of data, transferring, from the source storage tier to the target storage tier, only those segments of the objects not already present in the target storage tier, and the transferring of the segments is performed by the worker nodes, and for each of the objects, placing metadata associated with that object in a bucket.

Cold tiering microservice for deduplicated data

Systems and methods for cleaning a storage system. A deduplicated storage system is cleaned by identifying structures that include dead or unreferenced segments. This includes processing recipes to identify the segments that are no longer part of a live object recipe. Then, the dead segments are removed. This is accomplished by copying forward the live segments and then deleting, as a whole, the structure that included the dead segments.

Scalable garbage collection for deduplicated storage

Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A similarity group verification path is selected by a controller module based upon detection of an object storage memory size condition. The similarity group verification path includes controller phases to verify whether objects have been corrupted without having to incur costly read operations.

Verification microservice for deduplicated object storage system

Disclosed is a system for storage management comprising: receiving capacity information within a predetermined time period from a plurality of storage in a first appliance pool and a second appliance pool, each having a plurality of storage systems; tracking a first capacity limit of the first storage system in the first appliance pool based on the capacity information; performing a first capacity prediction of the first storage system based on the tacked result of the first storage system, wherein the first capacity prediction indicating when the first storage system will be out of space based on the capacity information and the tracked result; generating a first recommendation indicating when an extra capacity of the first storage system should be added based on one or more attributes including the first capacity prediction; and presenting the first recommendation with one or more options.

Method and systems for recommending storage management actions based on capacities of storage systems

Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A similarity group verification path is selected based upon detection of an object storage memory size condition. The similarity group verification path is performed by one or more worker nodes. The similarity group verification path includes worker node phases that verify whether objects have been corrupted without having to incur costly read operations.

Verification microservice for a deduplicated object storage system

Systems and methods enabling garbage collection operations and normal system operations concurrently. Concurrent operations are performed by configuring a similarity group to permit garbage collection and normal operations. This may include creating a new subgroup in a similarity group for write and deduplication purposes such that an impacted subgroup can be cleaned.

Concurrently performing normal system operations and garbage collection

Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A testing tool inserts object corruptions in object storage. A verification path is performed by a controller module and one or more worker nodes. The testing tool verifies whether the executed verification path accurately detected the testing tool's inserted object corruptions.

Testing verification paths provided by a verification microservice for a deduplicated object storage system

Systems and methods for marking similarity groups impacted by a garbage collection operation are disclosed. Similarity groups are used to identify segments associated with objects in a computing system. Using deletion records that identify objects to be deleted, the similarity groups impacted by the deletion records can be identified. The live segments associated with the impacted similarity groups are also identified. This allows segments that are associated with the deleted objects and that are not associated with any live objects to be removed.

Marking impacted similarity groups in garbage collection operations in deduplicated storage systems

Described is a system for detecting corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A controller module and one or more worker nodes execute verification paths in concert with each other to identify object corruptions. The controller module estimates a number of worker nodes required for execution of different types of verification paths.

Verification microservice for a deduplicated object storage system with worker node estimation

Described is a system for a verification microservice engine for generating and deploying a controller module and one or more worker nodes to detect corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. A worker node(s) stores slice recipes into worker node memory that correspond to slice recipe names verified by a controller module. For each slice recipe, the worker node(s) detects whether slice recipe corruptions, similarity group corruptions and/or compression region corruptions exist.

Verification microservice for execution of a slice verification path by worker nodes

Described is a system for a verification microservice engine for generating and deploying a controller module and one or more worker nodes to detect corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. The controller module builds local versions of slice recipe names based on metadata available object recipes. The controller module verifies the accuracy of the metadata based on whether the locally built slice recipes names match slice recipe names in object storage.

Growing community of inventors

San Jose, CA, United States of America

Tipper Truong