International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Sebastien Vas

Jason K Resch

Zachary J Mark

Wesley B Leggette

Andrew D Baptist

Gary W Grube

Greg R Dhuse

Timothy W Markison

Ilya Volvovski

Yogesh Ramesh Vedpathak

Adam Michael Gray

Eric Gunnar Smith

Manish Motwani

Thomas Franklin Shirley, Jr

Thomas Darrel Cocagne

Andrew George Peake

Sanjaya Kumar

Bart Richard Clifone

Eric G Smith

A method includes storing a plurality of data in a storage system. A plurality of identifiers corresponding to the plurality of data is determined and the plurality of identifiers are stored in the storage system. A first set of integrity information corresponding to a first system storage level is generated for the plurality of data by performing a first set of cyclic redundancy checks and the first set of integrity information is stored in the storage system. A second set of integrity information corresponding to a second system storage level is generated for the plurality of data and the second set of integrity information is stored in the storage system.

Generating multiple sets of integrity information in a vast storage system

A method for execution by a computing device of a storage network begins by receiving a write request for a data object, determining a plurality of data regions for the data object, determining storage identification information for each data region of the plurality of data regions and generating a storage table for the data object that includes information sufficient to identify each data region of the plurality of data regions. For a first data region of the plurality of data regions the method continues by dividing the first data region into a plurality of data segments, and dispersed error encoding the plurality of data segments to produce a plurality of sets of encoded data slices. The method then continues by sending a write request for each encoded data slice of each set of encoded data slices of the plurality of sets of encoded data slices to the storage network, and when at least a write threshold number of write responses is received for each of the plurality of sets of encoded data slices the method ends by updating the storage table to indicate that the first data region is available for retrieval.

Storing a data object as data regions in a storage network

A method includes determining a plurality of identifiers based on a data retrieval request. Integrity information is generated based on determining the plurality of identifiers. Stored integrity information corresponding to the data retrieval request is compared with the integrity information. When the stored integrity information compares unfavorably with the integrity information, corruption associated with the plurality of identifiers is determined.

Utilizing integrity information to determine corruption in a vast storage system

A method includes receiving a data retrieval request. A plurality of identifiers are determined in accordance with the data retrieval request. Integrity information is generated based on determining the plurality of identifiers by performing a cyclic redundancy check. Stored integrity information corresponding to the data retrieval request is compared with the integrity information, where the stored integrity information was previously generated by performing the cyclic redundancy check. When the stored integrity information compares unfavorably with the integrity information, corruption associated with the plurality of identifiers is determined.

Utilizing integrity information in a vast storage system

A system includes a plurality of storage units, where one or more storage vaults is associated with the plurality of storage units and each storage vault of the one or more storage vaults represents a software-constructed grouping of storage units of the plurality of storage units. The software-constructed grouping of storage units stores encoded data slices. A data segment is encoded using an information dispersal algorithm to produce the encoded data slices. The system further includes a grid access manager that generates a data structure pertaining to the software-constructed grouping of storage units. A storage unit of the software-constructed grouping of storage units receives, from a client computer of the system, a request regarding the data segment, obtains, from the data structure, information regarding the request, determines whether the request is valid based on the information regarding the request, and when the request is valid, executes the request.

Validating requests based on stored vault information

A method includes encoding data via erasure coding to produce a plurality of data slices. The method further includes determining a plurality of identifiers corresponding to the data. The method further includes generating integrity information based on the plurality of identifiers by performing a cyclic redundancy check. The method further includes storing the plurality of data slices, the plurality of identifiers, and the integrity information in a storage system.

Generating integrity information in a vast storage system

To identify slice errors, a processing module of a computing device in a dispersed storage network (DSN) sends first list digest requests to at least first and second dispersed storage (DS) units. The requests indicates a first range of slice names to include in a first list digest. The processing module receives digest responses from the DS units, and compares the digest responses to determine whether they identify the same slices. If they do not identify the same slices, the processing module sends second list digest requests indicating a sub-range of the first range of slice names to include in second list digests. The sub-range continues to be narrowed until the processing module identifies at least one sub-range of slice names where a slice error exists.

Digest listing decomposition

A method begins by a dispersed storage (DS) processing module dividing a very large data object into a plurality of data regions and generating a data object storage tracking table that includes sections for identifying one or more data regions that are available or unavailable for retrieval. The method continues with the DS processing module dividing a first data region into data segments and disperse storage error encoding the data segments to produce sets of encoded data slices. The method continues with the DS processing module sending DSN write requests regarding storing the sets of encoded data slices to storage units and when at least a write threshold number of write responses is received for each of the sets of encoded data slices, updating the data object storage tracking table to indicate that the first data region is available for retrieval.

Utilizing data object storage tracking in a dispersed storage network

A method begins by a dispersed storage (DS) processing module of a storage unit receiving a write request for storing a data object that includes updated access control list (ACL) information. The method continues with the DS processing module determining whether the data object is a new data object or a revised version of an existing data object and determining write authority of the requesting entity based on information contained in a locally stored access control list. When the write request is regarding the revised version of the existing data object and the write authority includes authorization to issue the write request for the revised version of the existing data object and authorization to issue the updated ACL information, the method continues with the DS processing module storing the revised version of the existing data object and updating the access control list based on the updated ACL information.

Updating access control information within a dispersed storage unit

A method includes issuing, by a managing unit of a dispersed storage network (DSN), a vault access restriction to a set of storage units of the DSN regarding a logical storage vault supported by the set of storage units. While the vault access restriction is active, the method continues by receiving, by a storage unit of the set of storage units, a data access request to the vault. The method continues by determining, by the storage unit, whether the data access request is regarding a rebuilding operation. When the data access request is regarding a rebuilding operation, the method continues by processing, by the storage unit, the data access request. When data access request is not regarding a rebuilding operation, the method continues by denying, by the storage unit, the data access request.

Temporary suspension of vault access

A method for execution by a dispersed storage and task (DST) processing unit includes obtaining distributed storage and task network (DSTN) address range information for each memory device of a set of memory devices associated with a storage unit targeted for an upgrade; obtaining a list of slice names associated with selected slices stored in each memory device of the storage unit; retrieving the selected slices associated with the list of slice names; generating an integrity check value for the selected slices; storing one or more of: the integrity check value, the selected slices, or the list of slice names as original integrity information in a memory; and enabling an upgrade sequence of the DST execution unit.

Upgrading devices in a dispersed storage network

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method begins by identifying a data object to access within a DSN. The method continues by identifying a vault ID based on the data object. The method continues by obtaining an object ID based on the data object. The method continues by selecting at least one generation ID based on generation status. The method continues, for each generation ID, by generating at least one set of slice names using the vault ID, the generation ID, and the object ID. The method continues, for each set of slice names, by generating a set of slice access requests that includes the set of slice names and accessing the DSN utilizing the set of slice access requests.

Storing and retrieving mutable objects

A technique of rebuilding encoded data slices in a dispersed storage network when detecting a plurality of encoded data slices that require rebuilding, as may occur upon failure of a memory device storing the encoded data slices. A plurality of rebuilding resources capable for use to rebuild the plurality of data slices are determined and, based on one or more attributes associated with the determination, a rebuilding task is apportioned. The resulting rebuilding assignments are allocated to the plurality of rebuilding resources to rebuild the encoded data slices. The allocation of the rebuilding assignments permits more than one rebuilding resource and associated distributed storage units to rebuild the encoded data slices.

Distributed rebuilding of data in a dispersed storage network

A technique of rebuilding data slices in a dispersed storage network when detecting a plurality of data slices that require rebuilding. A plurality of rebuilding resources capable for use to rebuild the plurality of data slices are determined and based on an attribute associated with the determination, a rebuilding task is apportioned and the apportioned tasks are assigned to the plurality of rebuilding resources to rebuild the data slices. The apportionment of the tasks permit more than one rebuilding resource and associated distributed storage unit to perform the rebuild of the data slices.

Growing community of inventors

Sunnyvale, CA, United States of America

Sebastien Vas