International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Michael Colin Storm

Jason K Resch

Wesley B Leggette

Ilya Volvovski

Greg R Dhuse

Manish Motwani

Thomas Darrel Cocagne

Gary W Grube

Ravi V Khadiwala

Yogesh Ramesh Vedpathak

Timothy W Markison

Bruno Hennig Cabral

Andrew D Baptist

Bart Richard Cilfone

Thomas Franklin Shirley, Jr

Joseph M Kaczmarek

Anthony J Baldocchi

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, at the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein at least one missing encoded data slice was not included in the first subset and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data corresponds to a second subset of the set of encoded data slices that includes the at least one missing encoded data slice and wherein the locally decodable redundancy data is stored locally to the processing circuit; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Recovering missing data in a storage network via locally decodable redundancy data

A method for execution by a computing device to adjust efficiency of storing data in a storage network includes processing a data segment based on a storage approach to produce a processed data segment, where the storage approach is based on an estimated update frequency for the data segment. The method continues by storing the data segment in the memory of the storage network in accordance with the storage approach.

Dynamic data storage based on estimated update frequency

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Rebuilding missing data in a storage network via locally decodable redundancy data

A method for execution by a computing device to adjust efficiency of storing data in a storage network includes processing a data segment based on a storage approach to produce a processed data segment, the storage approach being based on an estimated overwrite frequency for the data segment. The method continues by error encoding the processed data segment to produce a set of encoded data slices, where a decode threshold number of encoded data slices is needed to recover the processed data segment. The method continues by storing the set of encoded data slices in the memory of the storage network.

Adjusting efficiency of storing data

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of slices that is required to decode the data segment; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices, wherein the first subset of encoded data slices is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein the number of encoded data slices in the first subset of the encoded data slices is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

A method for execution by a computing device to adjust data storage efficiency of data in a storage network includes determining an estimated overwrite frequency for a data segment for storage in memory of the storage network. The method continues by determining a storage approach for the data segment based on the estimated overwrite frequency. The method continues by processing the data segment based on the storage approach to produce a processed data segment. The method continues by error encoding the processed data segment to produce a set of encoded data slices, where a decode threshold number of encoded data slices is needed to recover the processed data segment. The method continues by storing the set of encoded data slices in the memory of the storage network.

Adjusting efficiency of storing data in a storage network

A network storage system operates by: sending, to at least one storage unit of a storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is codable in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of slices that is required to decode the data segment; receiving from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices, wherein the first subset of encoded data slices is missing at least one missing encoded data slice from the number of the set of slices that is required to decode the data segment and that was not received from the at least one storage unit in response to the at least one read request; generating at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data includes a plurality of local redundancy slices generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Utilizing locally decodable redundancy data in a vast storage network

A method for updating software of storage units of a set of storage units in a storage network. In an embodiment, each storage unit of the set of storage units determines a data slice storage status for data slices stored in the storage unit (e.g., by monitoring rebuilding messages relating to the data slices). A processing module of the storage network obtains the data slice storage status of the storage units of the set of storage units and determines whether to perform a software update to software of the storage units of the set of storage units. Determining whether to perform the software update includes determining, based on the data slice storage status of the storage units, whether a threshold number of storage units are associated with a favorable data slice storage status. In response to determining that a threshold number of storage units are associated with a favorable data slice storage status, the processing module indicates to perform the software update.

Updating the configuration of storage units of a storage network

In various embodiments, a method is presented for execution by a processing circuit of a storage network device that includes a processor. The method comprises: receiving, via the processing circuit, a first subset of encoded data slices of a set of encoded data slices corresponding to a data segment; identifying, via the processing circuit, at least one missing encoded data slice of the first subset of slices that is required to decode the data segment; generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data; and recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Rebuilding data via locally decodable redundancy in a vast storage network

A method begins by a processing module of a dispersed storage network (DSN) determining a fault domain for a portion of the DSN and generating a local redundancy for the fault domain. The method continues with the processing module identifying storage locations available for storing the first local redundancy, selecting storage locations for storing the first local redundancy and continues with the processing module facilitating storage of the local redundancy in the storage locations. Upon detecting a storage failure in the DSN, the method continues with the processing module determining whether the storage failure is associated with the fault domain and in response determining whether the first local redundancy is associated with the first fault domain. In response to determining that the local redundancy is associated with the first fault domain the method continues with the processing module recovering the local redundancy and correcting the storage failure.

Reliably recovering stored data in a dispersed storage network

A method for execution by a computing device to adjust data storage efficiency of data in a storage network begins by obtaining a data segment for storage in memory of the storage network. The method continues by obtaining access level information regarding the data segment, where the access level information includes an estimated retrieval frequency level for the data segment. The method continues by determining a storage approach for the data segment based on the access level information and processing the data segment based on the storage approach to produce a processed data segment. The method continues by dispersed storage error encoding the processed data segment to produce a set of encoded data slices, where a decode threshold number of encoded data slices is needed to recover the processed data segment. The method continues by sending the set of encoded data slices to the memory for storage therein.

Adjusting data storage efficiency of data in a storage network

A method for execution by a dispersed storage and task (DST) execution unit includes identifying a plurality of pending operations. A resource availability level to support execution of at least one of the plurality of pending operations is determined, and a required resource level to execute the at least one of the plurality of pending operations is determined. A balance factor between at least two types of the plurality of pending operations is determined based on the resource availability level and the required resource level. Determination of required timing of the execution of the at least one of the plurality of pending operations is coordinated with at least one other DST execution unit. An operation execution schedule is updated based on the required resource levels, the resource availability level, the balance factor, and the required timing of the execution.

Establishing an operation execution schedule in a storage network

A method for execution by a processing system of a dispersed storage and task (DST) processing unit comprises: receiving one or more encoded data slices for storage; storing the one or more encoded data slices in one or more memories of a set of memories; determining a level redundancy for the one or more encoded data slices; generating redundancy information for the one or more encoded data slices in accordance with the level redundancy; storing the redundancy information in another one or more memories of the set of memories; determining to update the level of redundancy; determining an updated level of redundancy based on one or more of a storage utilization level and a storage reliability level; updating the redundancy information based on the updated level of redundancy and updating storage of the redundancy information based on the updated redundancy information.

Reliable storage in a dispersed storage network

A method for execution by a resource allocation module includes facilitating migration of a first set of encoded data slices stored at a storage unit for decommissioning to a newly commissioned storage unit, and facilitating migration of a remaining set of encoded data slices stored at the storage unit for decommissioning as foster encoded data slices to at least one other storage unit. For each foster encoded data slice, it is determined whether to facilitate migration of the foster encoded data slice to the newly commissioned storage unit. When determining to facilitate the migration of the foster encoded data slice, the migration of the foster encoded data slice to the newly commissioned storage unit is facilitated. An association of the newly commissioned storage unit and identity of the foster encoded data slice is updated in response to detecting successful migration of the foster encoded data slice.

Modifying allocation of storage resources in a dispersed storage network

A method includes determining, by a computing device of a dispersed storage network (DSN), a source name for a data object to be scanned for missing encoded data slices. The method further includes issuing list source requests to the set of storage units. When a list source response is not received from a storage unit of the set of storage units within a response timeframe, the method further includes identifying one or more encoded data slices stored on the storage unit as potentially missing encoded data slices; determining a next level missing encoded data slice determination approach for the storage unit based on one or more of: a number of potentially missing encoded data slices, a performance goal, a network loading level, a rebuilding loading level, a predetermination, and an entry of a system registry; and executing the next level missing encoded data slice determination approach.

Growing community of inventors

Chicago, IL, United States of America

Michael Colin Storm