International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Yogesh Ramesh Vedpathak

Jason K Resch

Wesley B Leggette

Ravi V Khadiwala

Greg R Dhuse

Manish Motwani

Ilya Volvovski

Gary W Grube

Thomas Darrel Cocagne

Michael Colin Storm

Timothy W Markison

Bruno Hennig Cabral

Andrew D Baptist

Daniel J Scholl

Bart Richard Cilfone

Thomas Franklin Shirley, Jr

Adam Michael Gray

Harsha Hegde

Rohan P Shah

Asimuddin Kazi

Sebastien Vas

Renars W Narubin

Joseph M Kaczmarek

Eric Gunnar Smith

Jeremy S Jarczyk

Thomas Dubucq

Mingyu Li

Ramin Rouzbeh

Mark Duane Seaborn

Tyler K Reid

Brian S Farrell

Eric G Smith

Lakshmi Krishna Priya Kala

A method for execution by one or more processing modules of a storage network begins by receiving an access request for a set of encoded data slices, where the data object is segmented into a plurality of data segments, a data segment of which is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of encoded data slices. The method continues by determining whether a revision for one or more encoded data slices in a first storage module is a most recent revision level for the one or more encoded data slices and when the one or more encoded data slices is a most recent revision level, determining whether the revision level for the one or more encoded data slices in a second storage module is the most recent revision level. When the one or more encoded data slices in the second storage module is not the most recent revision level the method continues by facilitating a read request to the first storage module for the one or more encoded data slices and synchronizing the one or more encoded data slices in the second storage module with the one or more encoded data slices in the first storage module.

Managing copy revisions in a distributed storage system

A method for execution by a computing device of a storage network begins by receiving a write request for a data object, determining a plurality of data regions for the data object, determining storage identification information for each data region of the plurality of data regions and generating a storage table for the data object that includes information sufficient to identify each data region of the plurality of data regions. For a first data region of the plurality of data regions the method continues by dividing the first data region into a plurality of data segments, and dispersed error encoding the plurality of data segments to produce a plurality of sets of encoded data slices. The method then continues by sending a write request for each encoded data slice of each set of encoded data slices of the plurality of sets of encoded data slices to the storage network, and when at least a write threshold number of write responses is received for each of the plurality of sets of encoded data slices the method ends by updating the storage table to indicate that the first data region is available for retrieval.

Storing a data object as data regions in a storage network

A method includes writing sets of encoded data slices to storage units of a storage network in accordance with error encoding parameters, where for a set of encoded data slices, the error encoding parameters include an error coding number and a decode threshold number, the error coding number indicates a number of encoded data slices that results when a data segment is encoded using an error encoding function and the decode threshold number indicates a minimum number needed to recover the data segment. The method further includes monitoring processing of the writing the sets of encoded data slices to produce write processing performance information. When the write processing performance information compares unfavorably to a desired write performance range, the method further includes adjusting at least one of the error coding number and the decode threshold number to produce adjusted error encoding parameters for writing subsequent encoded data slices.

Adjusting error encoding parameters for writing encoded data slices

A method includes receiving an error message including first information associated with a first reported error and a time at which the first reported error was detected. First stored event records associated with second reported errors are identified. The first stored event records include second information describing previously reported errors that occurred within a predetermined time prior to the time at which the first reported error was detected. The method determines, based on the first information and the second information, whether a correlation exists among one or more of the previously reported errors and the first reported error. In response to determining that the correlation exists, generating an error correlation report predicting occurrence of a third error.

Error prediction based on correlation using event records

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Rebuilding missing data in a storage network via locally decodable redundancy data

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of slices that is required to decode the data segment; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices, wherein the first subset of encoded data slices is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein the number of encoded data slices in the first subset of the encoded data slices is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Recovering missing data in a storage network via locally decodable redundancy data

A method begins by a storage unit of a dispersed storage network (DSN) receiving access requests which include a logical DSN address and a storage function. The method continues with a first processing module of the storage unit performing logical to physical address conversions of logical DSN addresses of the access requests to physical addresses of a plurality of main memories. For a first access request of the access requests, the method continues with the first processing module identifying a first main memory based on the physical address resulting from the physical address conversion. The method continues with the first processing module identifying a first processing thread of a plurality of processing threads based on allocation of the plurality of processing threads to the plurality of main memories. The method continues with the first processing thread executing tasks of the first access request to fulfill the first access request.

Associating a processing thread and memory section to a memory device

Event records from multiple computing devices are received at a managing unit. Individual event records include an event identifier field including an event identifier identifying a first event associated with a particular computing device, a parent event identifier field identifying a parent event that initialized the first event, and an entity identifier field including an entity identifier identifying the particular computing device. The managing unit generates log records associated with event identifiers included in the event records. The log records include state fields indicating a state of a particular event associated with a particular event identifier. Based on a correlation of the event and log records, the managing unit determines at least two computing devices associated with events resulting in an error state, and identifies parent events that initialized the events with errors. The managing unit generates a report linking the parent events to the events having an error state.

Identifying a parent event associated with child error states

A storage unit includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry. The storage unit receives a set of read slice requests for a set of encoded data slices (EDSs) associated with a data object stored within a first set of storage units, where the storage the first set of storage units includes the storage unit. When at least a read threshold number of EDSs and fewer than all of the set of EDSs can be successfully retrieved from the first set of storage units, the storage unit identifies at least one EDS associated with a data object that is stored in a second set of storage units, obtains the at least one EDS and stores the at least one EDS in the storage unit.

Using copied data in a distributed storage network

A network storage system operates by: sending, to at least one storage unit of a storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is codable in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of slices that is required to decode the data segment; receiving from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices, wherein the first subset of encoded data slices is missing at least one missing encoded data slice from the number of the set of slices that is required to decode the data segment and that was not received from the at least one storage unit in response to the at least one read request; generating at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data includes a plurality of local redundancy slices generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Utilizing locally decodable redundancy data in a vast storage network

In various embodiments, a method is presented for execution by a processing circuit of a storage network device that includes a processor. The method comprises: receiving, via the processing circuit, a first subset of encoded data slices of a set of encoded data slices corresponding to a data segment; identifying, via the processing circuit, at least one missing encoded data slice of the first subset of slices that is required to decode the data segment; generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data; and recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Rebuilding data via locally decodable redundancy in a vast storage network

Systems and methods for replicating containers in object storage using intents are disclosed. A DS processing unit, upon reception of a write request may determine the location of replicated instances for the container. DS processing unit can then generate an intent for each fork, to indicate a specific write request received. The DS processing unit may save or persist intent objects within one of the containers, or in a dispersed data structure. A DS unit or DS processing unit may then check for intents whose operation has not been completed, and when resources to perform the operation are available, perform the operation specified in the intent. DS processing unit can remove a pending intent once the job or operation is complete. A clean-up agent could run periodically to complete unfinished jobs that are pending within DS processing unit or DS unit due to containers not being available for whatever reason.

Replicating containers in object storage using intents

A method includes determining, by a storage unit of a dispersed storage network (DSN), a storage unit memory pressure level. When the storage unit memory pressure level compares unfavorably to a threshold, the method further includes, in response to a data access request regarding an encoded data slice from a computing device, determining whether the data access request includes an override message or a non-override message. When the data access request includes the non-override message, the method includes generating a storage unit memory pressure level message in accordance with the storage unit memory pressure level and the type of data access request, sending the storage unit memory pressure level message to the computing device, and processing the data access request in accordance with the storage unit memory pressure level message. When the data access request includes the override message, the method further includes processing the data access request.

Processing data access requests in accordance with a storage unit memory pressure level

A managing unit included in a distributed storage network (DSN) receives an event representation request, and identifies event record entries based on that request. The event record entries include information associating reporting entities with the event record entries. The management unit obtains the event record entries from the reporting entities; at least one event record entry is obtained from a first reporting entity, and at least another event record entry is obtained from a second reporting entity. In response to receiving the event representation request, the management unit generates a representation of the event record entries, and outputs the representation to a requesting entity.

Hierarchical event tree

A method begins by a processing module of a dispersed storage network (DSN) determining a fault domain for a portion of the DSN and generating a local redundancy for the fault domain. The method continues with the processing module identifying storage locations available for storing the first local redundancy, selecting storage locations for storing the first local redundancy and continues with the processing module facilitating storage of the local redundancy in the storage locations. Upon detecting a storage failure in the DSN, the method continues with the processing module determining whether the storage failure is associated with the fault domain and in response determining whether the first local redundancy is associated with the first fault domain. In response to determining that the local redundancy is associated with the first fault domain the method continues with the processing module recovering the local redundancy and correcting the storage failure.

Growing community of inventors

Chicago, IL, United States of America

Yogesh Ramesh Vedpathak