Cleversafe, Inc.

International Business Machines Corporation

Pure Storage, Inc.

Purage Storage, Inc.

John Quigley

Jason K Resch

Greg R Dhuse

Andrew D Baptist

Vance T Thornton

Ilya Volvovski

Wesley B Leggette

Zachary J Mark

Srinivas Palthepu

S Christopher Gladwin

Manish Motwani

Bart Richard Cilfone

Asimuddin Kazi

Dustin M Hendrickson

Jamie Bellanca

Jesse Louis Young

Trent William Johnson

Dusty Hendrickson

Akshay Lal

Lynn Foster

Mainsh Motwani

Christopher S Gladwin

A processor in a storage network operates by: receiving an access request for a data segment, wherein the data segment is encoded utilizing an error correcting information dispersal algorithm as a set of encoded data slices that are stored in a plurality of storage units of the storage network and wherein each encoded data slice of the set of encoded data slices includes a corresponding checksum of a plurality of checksums; retrieving, from the storage network, a subset of encoded data slices that includes a threshold number of encoded data slices of the set of encoded data slices; determining, based on ones of the plurality of checksums corresponding to the subset of encoded data slices, when the subset of encoded data slices includes at least one corrupted encoded data slice; retrieving from at least one of the plurality of storage units an addition number of encoded data slices required to generate a reconstructed data segment based on the subset of encoded data slices; generating the reconstructed data segment in accordance with the error correcting information dispersal algorithm, using the additional number of encoded data slices and at least some of the subset of encoded data slices; providing the reconstructed data segment in response to the access request; forming a reconstructed set of encoded data slices utilizing the error correcting information dispersal algorithm on the reconstructed data segment; and replacing the at least one corrupted encoded data slice with at least one reconstructed encoded data slice of the reconstructed set of encoded data slices.

Data reconstruction in a storage network and methods for use therewith

Reconstructing data segments in a storage network and methods for use therewith

A storage unit in a distributed storage network (DSN) transmits resource availability information to a managing unit. The resource availability information includes information indicating a number of available memory devices of the storage unit that are available to be used for storage of encoded data slices. The storage unit receives resource assignment information determined by the managing unit. The resource assignment information includes one or more dispersal parameters, information identifying a set of storage units to be included in a newly instantiated DSN memory, and a DSN address range assigned to the storage unit for use in the newly instantiated DSN memory. The storage unit selects particular available memory devices of the storage unit to assign to the newly instantiated DSN memory. The selection is based on the resource assignment information.

Storage unit selection of memory devices used for distributed storage network memory

A method with a computer generating a first set of access requests regarding a first set of encoded data slices and storage units of a dispersed storage network. A first data segment is encoded into the first set of encoded data slices. The method continues with the computer generating a second set of access requests regarding a second set of encoded data slices and the storage units. A second data segment is encoded into the second set of encoded data slices. The method continues with the computer grouping the first set of access requests and the second set of access requests to produce a set of combined requests. The method continues with the computer sending the set of combined requests to the storage units.

Integrated client for use with a dispersed data storage network

A dispersed storage (DS) processing module may include a gateway module operable to communicate data and/or corresponding information with a user device and may include an access module operable to segment outbound data of the data into one or more outbound data segments and aggregate one or more inbound data segments into inbound data of the data. The DS processing module may include a grid module operable to encode an outbound data segment of the one or more outbound data segments into a plurality of outbound encoded data slices and decode a plurality of inbound encoded data slices into an inbound data segment of the one or more inbound data segments. The DS processing module may include a storage module operable to output the plurality of outbound encoded data slices to a plurality of DS storage units and receive the plurality of inbound encoded data slices from the plurality of DS storage units.

Distributed storage processing module

A managing unit included in a distributed storage network (DSN) determines resource availability information. The resource availability information includes information indicating a number of memory devices available to be used for storage of encoded data slices. The management unit then chooses dispersal parameters based on the resource availability information. The dispersal parameters are chosen to satisfy a performance threshold associated with storage of the encoded data slices in the DSN, but subject to a constraint requiring a pillar width associated with the encoded data slices to be no larger than the number of memory devices available. The management unit creates a number of storage units, wherein the number of storage units is constrained to be greater than the pillar width, and wherein one or more of the number of memory devices is assigned to each of the number of storage units. The management unit then instantiates a DSN memory that includes the number of storage units.

Meeting storage requirements with limited storage resources

An improved system is disclosed for ensuring the integrity of data stored on a dispersed data storage network. Checksums are used to ensure integrity of both data segments and data slices. Checksums appended to data slices are checked by receiving slice servers to ensure that no errors occurred during transmission. Slice servers also periodically recalculate checksums for stored data slices to ensure that data slices have not been corrupted during storage. Checksums appended to data segments are checked when data segments are read from the storage network.

Ensuring data integrity on a dispersed storage grid

For each original data segment, a distributed storage processing unit generates encoded slices designed to prevent the original data segment from being reconstructed using fewer than a threshold number of encoded slices. Multiple encoded slices are generated for each of two different data segments, and the slices associated with the first and second data segment are stored substantially concurrently in different storage sets employing different distributed storage units. Encoded slices for even and odd data segments can be stored in different storage sets, or longer sequences of data segments can be stored in alternating storage sets. Storage sets can also be determined by the vault generation of a particular data segment.

Concurrent set storage in distributed storage network

A computer includes a data transform algorithm, a data dispersal algorithm, and a network port. The data transform algorithm performs a data transformation on a data block to produce a transformed data block. The data dispersal algorithm performs a data dispersal function on the transformed data block to produce a plurality of data slices, wherein each of the plurality of data slices includes less than all data contained in the transformed data block. The network port is operable to transmit a plurality of write commands to a plurality of slice servers, wherein each of the plurality of write commands includes a corresponding one of the plurality of data slices. The network port is further operable to receive verification of storage of at least some of the plurality of data slices from at least some of the plurality of slice servers.

Block based access to a dispersed data storage network

A method to generate a protocol header of a dispersed storage network (DSN) frame begins by a processing module generating a protocol class field and generating a protocol class version field. The method continues with the processing module generating an operation code (opcode) field and generating a request/response field. The method continues with the processing module generating a request number field and generating a payload length field. The method continues with the processing module outputting, in order, the protocol class field, the protocol class version field, the opcode field, the request/response field, the request number field, and the payload length field.

Dispersed storage network frame protocol header

A method begins by a processing module dispersed storage error encoding data to produce a plurality of sets of encoded data slices in accordance with dispersed storage error coding parameters. The method continues with the processing module determining a plurality of sets of slice names corresponding to the plurality of sets of encoded data slices. The method continues with the processing module determining integrity information for the plurality of sets of slice names and sending the plurality of sets of encoded data slices, the plurality of sets of slice names, and the integrity information to a dispersed storage network memory for storage therein.

Dispersed storage network slice name verification

A method begins by a processing module obtaining common storage name information regarding data that is stored in storage units of a distributed storage network (DSN) as a set of data slices. Each data slice of the set of data slices has a unique storage name, where each of the unique storage names for the set of data slices has common naming information regarding the data. The method continues where the processing module interprets the common storage name information to determine whether a difference exists between the common naming information of a data slice of the set of data slices and the common naming information of other data slices of the set of data slices. When the difference exists, the method continues where the processing module indicates a potential storage error of the data slice and implements a storage error process regarding the potential storage error of the data slice.

Identifying a storage error of a data slice

An integrated client computer for accessing a dispersed data storage network is disclosed. A client program operating on the client computer accepts commands from a user and in response, issues network messages to a plurality of slice servers implementing a dispersed data storage network. Data slices can be read, written or modified using the disclosed integrated client computer.

A data object interface to a dispersed data storage network is disclosed. The disclosed interface accepts read and write commands from a client computer and generates network commands that are forwarded to slice servers that form the storage component of the dispersed data storage network. The slice servers then fulfill the read and write commands.

Growing community of inventors

Chicago, IL, United States of America

John Quigley