Western Digital Technologies, Inc.

Netapp, Inc.

Ameet Pyati

Tomy Ammuthan Cheru

Satish Singhal

Carl Rene D'Halluin

Pawan Saxena

Abhishek Naidu

Vibhor Arunkumar Patale

Souvik Roy

Muhammad Tanweer Alam

Vivek Gupta

Example object storage systems, bookkeeping engines, and methods provide quota usage monitoring for control entities, such as accounts, users, and buckets. An object data store is configured to enable control entities to access data objects associated with each control entity. Data objects are mapped to the control entities and the data objects are processed to identify object usage values corresponding to each combination of data object and control entity. Total usage values are calculated for each control entity and used to determine a data object access response for a target data object and associated control entities.

Object storage system with control entity quota usage mapping

Example object storage systems, meta object generators, and methods provide versioned meta objects for internal operational data that may be replicated between data object stores. A meta object may be generated that includes meta object data, such as internal operational data. A meta object identifier may be assigned to the meta object. A first version identifier may be associated with the meta object data and a second version identifier may be associated with a marker for the meta object, where the marker prevents exposure as a user data object.

Object storage system with versioned meta objects

Example object storage systems, replication managers, and methods provide meta objects for priority replication of configuration data between data object stores. A meta object may be generated that includes configuration data. Data objects are selected for replication from a first replication queue. Meta objects associated with the data objects in the first replication queue are replicated, independent of the first replication queue, between object data stores.

Object storage system with priority meta object replication

Example object storage systems, meta object generators, and methods provide meta objects for replication of configuration data between data object stores. A meta object may be generated that includes configuration data. A meta object identifier may be assigned to the meta object. The meta object may be stored in a first object store with related data objects and may be replicated to a second object data store with the related data objects.

Object storage system with meta object replication

The described methods, systems, and other aspects can advantageously provide balanced multi-stage processing of non-uniform object data. An example method may receive a list of buckets. Each of the buckets in the list of buckets can store one or more restorable objects. The method further comprises distributing the list of buckets to the two or more second nodes; determining a number of the one or more restorable objects in each bucket; determining a size of the one or more restorable objects in each bucket; generating batches of to-be-restored data objects based on the determined number of the one or more restorable objects in each bucket and the determined size of the one or more restorable objects in each bucket; and distributing the batches among the two or more second nodes for storage-related task processing.

Distributed batch processing of non-uniform data objects

These methods, systems, and other aspects can provide for adaptive distribution/re-distribution of data processing tasks. An example method may create, by a processor in a first node, a status object for a data processing task and assign, by the processor in the first node, the data processing task to a worker of a second node. Further, after a first period of time, the method may query, by the processor in the first node, the status object for a task status of the assigned data processing task reported by the second node. The task status reflects a status of execution of the assigned data processing task by the second node. The method determines, by the processor in the first node, to reassign the assigned data processing task based on a received task status reported by the second node, and reassign, by the processor in the first node, the assigned task.

Adaptive work distribution in distributed systems

Example object storage systems and methods provide priority metadata processing. Metadata operations are received in response to change events for at least one data object. The metadata operations may include system operations configured to manage changes to data objects and user-method operations configured to execute user-defined methods using the data objects. System operations are executed with a first priority in response to system operations with the first priority being available for processing. User-method operations are executed with a second priority in response to no metadata operations with the first priority being available for processing.

Object storage system with metadata operation priority processing

A hardware and/or software facility to enable emulated storage devices to share data stored on physical storage resources of a storage system. The facility may be implemented on a virtual tape library (VTL) system configured to back up data sets that have a high level of redundancy on multiple virtual tapes. The facility organizes all or a portion of the physical storage resources according to a common store data layout. By enabling emulated storage devices to share data stored on physical storage resources, the facility enables deduplication across the emulated storage devices irrespective of the emulated storage device to which the data is or was originally written, thereby eliminating duplicate data on the physical storage resources and improving the storage consumption of the emulated storage devices on the physical storage resources.

Eliminating duplicate data by sharing file system extents

Method and system for managing storage units are provided. A free space module scans a storage unit data structure and a reference data structure to generate an intermediate data structure that identifies storage units that are not referenced by any storage unit client. A lookup module is initiated and the storage unit clients are notified that all new references to any storage unit should be verified with the lookup module. The free space module then verifies if any of the storage units in the intermediate data structure have been referenced since the intermediate data structure was created. Any referenced storage units are removed from the intermediate data structure and a data structure identifying unreferenced storage units is generated. The data structure is then used to allocate the identified storage units.

Growing community of inventors

Bengaluru, India

Ameet Pyati