Ebay, Incorporated

Formation Data Systems, Inc.

Mark S Lewis

Anna Povzner

Vinay Pundalika Rao

Robert J Gillen

Subramanian Lakshmanan

Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Data storage space recovery via compaction and prioritized recovery of storage space from partitions based on stale data

Data storage space recovery

A data location table master system generates a master data location table storing associations of tokens with storage nodes for varying responsibility levels. When the master data location table is updated, the data location table master system updates storage nodes affected by the update as well as other storage nodes and application nodes in the system. Then, the storage nodes and the application nodes store a copy of the master data location table. A token migration and synchronization process reallocates data object storage among the storage nodes based on the updated master data location table.

Assigning storage responsibility in a distributed data storage system with replication

A write request for the data object is received. The data object is stored at one or more of a plurality of storage nodes according to the write request. A storage manager catalog that maps a data object identification (DOID) for the data object is updated with an actual storage location of the data object, the DOID for a corresponding data object being calculated based on content of the data object, a plurality of data objects being organized into one or more storage volumes. The storage manager catalog includes a count of a number of instances of a data object that is currently referenced for each of the one or more storage volumes.

A storage cluster in a computing network can operate according to a service-level agreement (e.g., a set of performance conditions). The network can receive a request to register a new storage volume with the storage cluster. In response, the network may evaluate the current extent of replication of data in the storage cluster and how adding the new volume may affect the performance of the cluster to determine whether to register the new volume with the storage cluster.

Volume admission control for a highly distributed data storage system

New storage volumes are registered to a data storage environment. Registering new storage volumes is controlled based on the performance requirements of the storage volumes compared to the capacity of the data storage environment.

Volume admission control for high-performance distributed data storage system

Data objects are stored on storage devices, taking into account service level agreements or other quality of service parameters. In one aspect, data objects grouped into storage volumes. In addition, the storage devices are classified into different level storage tiers, where higher level storage tiers have higher performance and lower level storage tiers have lower performance. Ranks for the data objects are calculated, based on both a data usage pattern for the data object (e.g., recency and frequency) and on quality of service (QOS) parameters for the storage volume containing the data object. Examples of QOS parameters include service level agreements, priority, minimum and maximum input/output operations per second. The data objects are then stored on storage devices, based on the data objects' ranks and the storage devices' storage tiers.

Dynamic storage tiering based on performance SLAs

An input/output (I/O) request is dispatched. A determination is made regarding a storage volume to service. A determination is made regarding whether an actual disk throughput exceeds a first threshold rate. The first threshold rate exceeds a reserved disk throughput. Responsive to determining that the actual disk throughput exceeds the first threshold rate, a first storage volume is selected based on credits or based on priority. Responsive to determining that the actual disk throughput does not exceed the first threshold rate, a second storage volume is selected based on guaranteed minimum I/O rate. An I/O request queue associated with the determined storage volume is determined. An I/O request is retrieved from the determined I/O request queue. The retrieved I/O request is sent to a persistence layer that includes the selected storage volume.

Efficient scalable I/O scheduling

A method of providing secure storage and access by authorized users of a multi-tenant data storage system that contains deduplicated data; wherein a unique identifier is assigned to a given data object within a data volume catalog; further wherein the unique identifier includes 'ownership' of the data object within the object store itself. Each time a data object is stored by a user, the method of the present invention creates an entry within a data association table of the data storage system, which entry denotes the data volume catalog that has executed the data object write. Using the method of the present invention, only a user of a data volume catalog that has actually 'written' the data object will be able to access and read the data object. Thereby the method of the present invention affords data security for a multi-tenant, deduplicated data storage system.

Growing community of inventors

Pleasanton, CA, United States of America

Mark S Lewis