International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Other

Purage Storage, Inc.

Ilya Volvovski

Jason K Resch

Greg R Dhuse

Andrew D Baptist

Wesley B Leggette

Manish Motwani

Gary W Grube

Ravi V Khadiwala

S Christopher Gladwin

Zachary J Mark

Timothy W Markison

Ethan S Wozniak

Praveen Viraraghavan

Thomas Franklin Shirley, Jr

Michael Colin Storm

Bart Richard Cilfone

Thomas Darrel Cocagne

Adam Michael Gray

Yogesh Ramesh Vedpathak

Vance T Thornton

Bruno Hennig Cabral

Jamie Bellanca

Scott Michael Horan

John Quigley

Joseph M Kaczmarek

Jordan Harrison Williams

Niall John McShane

Trevor J Vossberg

Ahmad Alnafoosi

Benjamin Lee Martin

Mark Duane Seaborn

Dustin M Hendrickson

Ying Zhao Guo

Daniel J Scholl

Sebastien Vas

Srinivas Palthepu

Tyler K Reid

Harsha Hegde

Andrew George Peake

Trent William Johnson

Sarah Toledano

Lynn Foster

Anthony J Baldocchi

Asimuddin Kazi

Teague Scott Algie

Shaorong Liu

Randy D Pfeifer

Dusty Hendrickson

Mainsh Motwani

Patrick Aaron Tamborski

Gregory R Dhuse

Renars W Narubin

Bart Richard Clifone

Jerri Lynn Foster

James Bellanca

Khushbu Patel

Vance T Thorton

A processor in a storage network operates by: receiving an access request for a data segment, wherein the data segment is encoded utilizing an error correcting information dispersal algorithm as a set of encoded data slices that are stored in a plurality of storage units of the storage network and wherein each encoded data slice of the set of encoded data slices includes a corresponding checksum of a plurality of checksums; retrieving, from the storage network, a subset of encoded data slices that includes a threshold number of encoded data slices of the set of encoded data slices; determining, based on ones of the plurality of checksums corresponding to the subset of encoded data slices, when the subset of encoded data slices includes at least one corrupted encoded data slice; retrieving from at least one of the plurality of storage units an addition number of encoded data slices required to generate a reconstructed data segment based on the subset of encoded data slices; generating the reconstructed data segment in accordance with the error correcting information dispersal algorithm, using the additional number of encoded data slices and at least some of the subset of encoded data slices; providing the reconstructed data segment in response to the access request; forming a reconstructed set of encoded data slices utilizing the error correcting information dispersal algorithm on the reconstructed data segment; and replacing the at least one corrupted encoded data slice with at least one reconstructed encoded data slice of the reconstructed set of encoded data slices.

Data reconstruction in a storage network and methods for use therewith

A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Storing encoded data slices in primary storage slots

A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Rebuilding data slices in a storage network based on priority

A processing system is operable to encode data to produce a first set of data slices based on a value of a width parameter. The data is stored based on maintaining storage of the first set of data slices across a set of storage units of a storage pool. Storage of the first set of data slices is maintained in the set of storage units of the storage pool after addition of an additional set of storage units added to the storage pool. The value of the width parameter is increased to an increased value to produce an updated width parameter. The data is re-encoded in accordance with the updated width parameter to produce a second set of data slices. The data is re-stored based on maintaining storage of the second set of data slices across the expanded set of storage units of the storage pool.

Re-encoding data in a storage network based on addition of additional storage units

A method for execution by one or more computing devices includes selecting a first routing path from a plurality of routing paths to a set of storage units based on routing path performance information, where the first routing path has a performance level greater than a first performance threshold. The method further includes selecting a second routing path from the plurality of routing paths based on the routing path performance information, where the second routing path has a performance level less than or equal to the first performance threshold. The method further includes sending a first subset of encoded data slices to the set of storage units via the first routing path for storage therein. The method further includes sending a second subset of encoded data slices to the set of storage units via the second routing path for storage therein.

Sending encoded data slices via multiple routing paths

A storage network operates by: storing at least one data object in the storage network as encoded data slices that are dispersed error encoded, the encoded data slices including locally decodable code segments; detecting at least one storage error associated with storage of at least one of the encoded data slices in the storage network; and recovering and at least one of the encoded data slices utilizing the locally decodable code segments.

Recovering a data segment using locally decodable code segments

A data segment is encrypted to produce an encrypted data segment, and a data tag associated with the data segment is generated. The encrypted data segment is encoded to generate a set of encoded data slices. At least a read-threshold number of encoded data slices are required to reconstruct the encrypted data segment. A set of write slice requests, which includes the set of encoded data slices and the data tag, is transmitted to a DSN memory. A determination is made, based on the data tag, whether a first encoded data slice of the set of encoded data slices is a duplicate of a second encoded data slice already stored within the DSN memory. If it is a duplicate, rather of storing the first encoded data slice, a reference to a location of the second encoded data slice is stored.

Efficient storage of data in a dispersed storage network

A method includes receiving, by a storage unit of a storage network, a check request message from a computing device of the storage network, where the check request message includes a group of slice names of a plurality of sets of slice names associated with a plurality of sets of encoded data slices, where a plurality of data segments are dispersed storage error encoded into the plurality of sets of encoded data slices. The method further includes determining, by the storage unit, whether an error condition exists for an encoded data slice associated with the group of slice names. When no, the method includes sending, by the storage unit, a check response message to the computing device, where the check response message includes a group of slice information regarding the group of slice names. When yes, the method includes disregarding sending the check response message.

Check requests in a storage network

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Rebuilding missing data in a storage network via locally decodable redundancy data

A computing device includes an interface configured to interface and communicate with a storage network, a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice associated with a data object, determines whether the encoded data slice is stored in the first memory and in response to a determination that the encoded data slice is not stored in the first memory, issues another data access request for the encoded data slice to a second memory, where the first memory includes access characteristics that are faster than the second memory. When a data access response including the encoded data slice is received from the second memory, a response including the encoded data slice is transmitted.

Utilizing memories of different operational speeds in a vast storage network

Reconstructing data segments in a storage network and methods for use therewith

A method includes determining an information dispersal algorithm width number and determining a number of primary storage slots from a number of storage slots associated with a set of storage units deployed across multiple sites, where the number of primary storage slots is equal to or greater than the information dispersal algorithm width number. The method further includes determining a mapping of primary storage slots to storage units. The method further includes sending configuration information to the set of storage units that includes the mapping. The method further includes storing a set of encoded data slices in the primary storage slots in accordance with the configuration information, where a data segment is error encoded into the set of encoded data slices in accordance with the information dispersal algorithm width number and a decode threshold number, which is a number of encoded data slices are needed to reconstruct the data segment.

Mapping storage slots to a set of storage units

Methods and apparatus for proxying data access requests in a storage network are disclosed. In various embodiments, a computing device associated with a first storage location of the storage network performs a method that includes receiving a portion of data that is in the process of being replicated from a second storage location of the storage network. The computing device facilitates storage of the portion of data in the first storage location of the storage network. In response to receiving a read request for the portion of data during the data replication process, the computing device determines whether the portion of data is available from memory of the first storage location. In response to determining that the portion of data is available from memory of the first storage location, the computing device issues a read response that includes the portion of data.

Proxying a data access request in a storage network

A method for execution by a storage network includes receiving a request pertaining to a data object. Metadata associated with the data object is determined and used to identify data segments associated with the data object and a set of storage units associated with the data segments. Based on a set of query requests, a response is received from a storage unit from the set of storage units. When the response indicates a storage environment that is unfavorable as compared to predetermined performance metrics, the storage network facilitates migration of encoded data slices associated with the storage unit to another storage unit.

Verifying and migrating data slices in a storage network

A method for execution by a computing device to adjust efficiency of storing data in a storage network includes processing a data segment based on a storage approach to produce a processed data segment, the storage approach being based on an estimated overwrite frequency for the data segment. The method continues by error encoding the processed data segment to produce a set of encoded data slices, where a decode threshold number of encoded data slices is needed to recover the processed data segment. The method continues by storing the set of encoded data slices in the memory of the storage network.

Adjusting efficiency of storing data

Growing community of inventors

Chicago, IL, United States of America

Ilya Volvovski