International Business Machines Corporation

Thomas Dubucq

Daniel J Scholl

Kevin Michael Freese

Jason K Resch

Manish Motwani

Ravi V Khadiwala

Yogesh Ramesh Vedpathak

Renars W Narubin

Jeremy S Jarczyk

Vivek Bajpai

Greg R Dhuse

Andrew D Baptist

Wesley B Leggette

Akila Srinivasan

David Brittain Bolen

A method includes: obtaining, by a computing device, a first work item from a first index, wherein the first work item represents a namespace of a bucket of a vault in a dispersed storage network; dividing, by the computing device, the namespace into plural ranges of names; creating, by the computing device, plural second work items, each respective one of the plural second work items including a respective one of the plural ranges of names; and adding, by the computing device, each of the plural second work items to a second index.

Namespace range creation to distribute workload in a dispersed storage system

A method includes: identifying, by a first program module, a bucket having an object lifecycle management rule in a dispersed storage network; creating, by a second program module, ranges of object names included in a namespace of the bucket; determining, by a third program module and for respective objects in each of the ranges, whether the respective objects satisfy the object lifecycle management rule of the bucket; and performing, by a fourth program module and based on the determining, an action on one of the respective objects that satisfies the object lifecycle management rule, the action being defined by the object lifecycle management rule.

Object lifecycle management in a dispersed storage system

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN). The method includes performing multiple splits in a single operation with a dispersed lockless concurrent index (DLCI) backed by a distributed transaction protocol by determining how many nodes n are required to hold data object entries, adding the n nodes to a distributed transaction, and when a parent node does not require a cascading split, adding it to the distributed transaction, and when the parent node needs to be split, recursively applying the determining and adding steps to the parent node until reaching a node that does not need to be split or is a root node of the DLCI.

Split-n and composable splits in a dispersed lockless concurrent index

A method, system, and computer program product for implementing indexes in a dispersed storage network (dsNet) are provided. The method accesses a work queue containing a set of work items as a set of key-value pairs. The key-value pairs are tuples including a work identifier and a work lease timestamp. The method selects a first work identifier and a first lease timestamp for a new work. The set of work items and the new work item are ordered according to a priority scheme to generate a modified work queue. Based on the modified work queue, the method transmits a work request to a plurality of data source units. The work request including a hash parameter and a bit parameter. The hash parameter is associated with a key-value pair of the modified work queue. The bit parameter indicates a number of bits of the hash parameter to consider.

Leasing prioritized items in namespace indices

A method, system, and computer program product for implementing indexes in a dispersed storage network (dsNet) are provided. The method accesses an unordered work queue containing a set of key-value pairs. A subset of expired key-value pairs are determined from the set of key-value pairs. The method transmits a work request to a plurality of data source units. The work request indicates a key-value pair being selected at random from the subset of expired key-value pairs. A threshold number of work responses is received from the plurality of data source units. The work responses include a set of available key-value pairs. The method selects an available key-value pair from the set of available key-value pairs and generates a work lease on the available key-value pair.

Leasing unordered items in namespace indices

A method, system, and computer program product for implementing indexes in a dispersed storage network (dsNet) are provided. The method receives a key-value list request including a start key. In response to the key-value list request, a set of key-value list requests are transmitted to a plurality of data source units within a dispersed storage network (dsNet). The method identifies a set of keys returned from the plurality of data source units and identifies a subset of keys from the set of keys returned. The subset of keys including a union of the set of keys. The key-value pairs associated with the subset of keys are restored. The method generates a key-value list response to the key-value list request. The key-value list response includes the restored key-value pairs associated with the subset of keys.

Listing and protocol for namespace index

A method, system, and computer program product for implementing indices in a dispersed storage network (dsNet) are provided. The method receives a key-value pair to be stored in a dsNet. The method routes the key and the value within a data source containing a SourceName repository and a data buffer. The key is routed to the SourceName repository and the value is routed to the data buffer. The data source is erasure encoded into a set of data slices having a slice name and a slice buffer. The method stores the set of data slices within the dsNet. The method generates a namespace index with an index entry for the key-value pair. The index entry represents the key-value pair as a SourceName and a data source indicator with the SourceName and the data source indicator being associated with the set of data slices.

Namespace indices in dispersed storage networks

Traversing nodes of a Dispersed Lockless Concurrent Index (DLCI), by retrieving a node look-up request from a request queue, executing the look-up request across nodes of a DLCI, storing the look-up request results in a results queue, and adding new look-up requests associated with any node pointers from the node(s) of the look-up request, in the request queue.

Traversal of dispersed lockless concurrent index nodes

A method includes registering, by a first computing device, with a distributed storage network (DSN) to indicate that a first storage site is connected via a common local area network with the first computing device and is the primary storage site for the first computing device. The method continues with the first computing device to store a write threshold number of encoded data slices (EDSs) to the first storage site, obtaining a first writing pattern for writing the write threshold number of EDSs to the first plurality of storage units, and transmitting at least a portion of a remaining number of EDSs to one or more storage units at a second storage site, where the remaining number of EDSs are EDSs not included in the write threshold number of EDSs and the second storage site and the first storage site are each one of a sharing group of storage sites.

Multiple writes using inter-site storage unit relationship

A method includes utilizing, by a first computing device, a first writing pattern to write a set of encoded data slices to a sharing group of sites. The first writing pattern includes writing a write threshold number of encoded data slices to storage units of a first site and writing a remaining number of encoded data slices to another storage unit(s) in another site(s). The method further includes sending, by storage units of the first site, encoded data slices of up to the write threshold number to other storage units in accordance with an inter-site storage unit relationship. The method further includes, when indicating in the inter-site storage unit relationship, sending, by storage unit(s) of the other site(s), encoded data slices of the remaining number of encoded data slices to other storage units in accordance with the inter-site storage unit relationship.

A storage unit (SU) includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the SU based on the operational instructions, is configured to perform various operations. The SU stores at least one encoded data slice (EDS) of first EDSs corresponding to a data object that are distributedly stored in first SUs and also an intent message that includes specifications for consistency between the data object and metadata of the data object. A second set of EDSs corresponding to the metadata are distributedly stored in second SUs. The SU services the intent message to determine consistency of the data object and the metadata based on the specifications and deletes the intent message when they are consistent.

Ensuring consistency between content and metadata with intents

A method beings by a computing device receiving a write request for a data segment that has been encoded to produce a set of encoded data slices. The method continues with the write request being stored in memory and a write intent associated with the write request being created and stored as an object in memory. The computing device then determines whether metadata associated with the data segment can be updated, and when the metadata cannot be updated maintaining the write request in memory until a cleanup agent can execute the write intent and successfully update the metadata.

Growing community of inventors

Chicago, IL, United States of America

Thomas Dubucq