International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Other

Wesley B Leggette

Jason K Resch

Andrew D Baptist

Greg R Dhuse

Ilya Volvovski

Manish Motwani

Gary W Grube

S Christopher Gladwin

Ravi V Khadiwala

Timothy W Markison

Bart Richard Cilfone

Thomas Franklin Shirley, Jr

Adam Michael Gray

Yogesh Ramesh Vedpathak

Michael Colin Storm

Scott Michael Horan

Ethan S Wozniak

Zachary J Mark

Thomas Darrel Cocagne

Daniel J Scholl

Bruno Hennig Cabral

Brian Frederick Ober

Franco Vincent Borich

Harsha Hegde

John Quigley

Trevor J Vossberg

Praveen Viraraghavan

Sebastien Vas

Rohan P Shah

Kumar Abhijeet

Vance T Thornton

Jamie Bellanca

Sarah Toledano

Alan E Holmes

Eric Gunnar Smith

James L Lester

Asimuddin Kazi

Dustin M Hendrickson

Srinivas Palthepu

Michael Edward Factor

Mark Duane Seaborn

Patrick Aaron Tamborski

Ram S Narayanan

Akila Srinivasan

Adam D Eggum

Lynn Foster

David Brittain Bolen

Jerri Lynn Foster

Vaijayanthimala K Anand

Niall John McShane

Ahmad Alnafoosi

Alan M Frazier

Thomas Dubucq

Tyler K Reid

Steven Mark Hoffman

Bart Richard Clifone

Jesse Louis Young

Kevin Michael Freese

Jeremy S Jarczyk

Bruno Henriques

Mainsh Motwani

Vivek Bajpai

David A Bukiet

Ilya Volvolvski

Ryan Joseph Kuester

Lakshmi Krishna Priya Kala

Greg R Ohuse

Cameron Paul Kurotori

Eric G Smith

Jr Thomas Franklin Shirley

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, at the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein at least one missing encoded data slice was not included in the first subset and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data corresponds to a second subset of the set of encoded data slices that includes the at least one missing encoded data slice and wherein the locally decodable redundancy data is stored locally to the processing circuit; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Recovering missing data in a storage network via locally decodable redundancy data

Methods and apparatus for redirecting data access requests in a storage network are disclosed. In an embodiment, a computing device associated with the storage network performs a method that includes identifying a (second) storage pool of the storage network to replace a first storage pool of the storage network. The method includes issuing a migration message to at least one of the first storage pool or the second storage pool. The migration message initiates a replication of data from the first storage pool to the second storage pool. The method further includes receiving a write data request for the first storage pool prior to completion of the replication of the data, and forwarding the write data request to the second storage pool for further processing.

Redirecting a data access request in a storage network

A method for execution by a computing device to adjust efficiency of storing data in a storage network includes processing a data segment based on a storage approach to produce a processed data segment, where the storage approach is based on an estimated update frequency for the data segment. The method continues by storing the data segment in the memory of the storage network in accordance with the storage approach.

Dynamic data storage based on estimated update frequency

Methods and apparatus for preference based selection of storage network memory for data storage. In an example, a computing device receives a data object for storage in memory of the storage network and determines a system level storage efficiency preference associated with the data object. The computing device selects a set of storage nodes of a plurality of sets of storage nodes for storage of the data object based, at least in part, on the system level storage efficiency preference. The computing device further determines dispersed storage error encoding parameters for the data object, encodes the data object in accordance with the dispersed storage error encoding parameters to produce encoded data slices, and generates system addressing information for the encoded data slices.

Preference based selection of storage network memory for data storage

A storage network operates by: issuing read slice requests to storage units of a set of storage units, where the read slice requests identify at least a read threshold number of encoded slices of a set of encoded slices corresponding to a data segment; when less than the read threshold number of encoded slices are received within a time threshold, facilitating receiving a decode threshold number of encoded slices of the set of encoded slices; decoding the decode threshold number of encoded slices to produce recovered encoded data slices, wherein a number of the recovered encoded data slices corresponds to the read threshold number minus a number of the encoded slices received within the time threshold; and outputting the recovered encoded data slices and the encoded slices of the read threshold number of encoded slices received within the time threshold.

Producing recovered data in a storage network

A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Storing encoded data slices in primary storage slots

A method for execution by one or more computing devices of a massive data ingestion system includes receiving data for storage in the massive data ingestion system. The method further includes storing a data segment of the data in a cache memory of the massive data ingestion system. The method further includes retrieving, based on one or more retrieval parameters, the data segment from the cache memory. The method further includes error encoding, in accordance with error encoding parameters, the data segment to produce a set of encoded data slices, wherein the error encoding parameters include a pillar width number and a decode threshold number, and wherein the decode threshold number is a minimum number of encoded data slices needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in a set of storage units of the massive data ingestion system.

Ingesting and storing data in a massive data ingestion system

A method for execution by a storage network begins by receiving data for storage by the storage network and continues by determining data preparation tasks for the data. The method continues by indexing the data in accordance with the data preparation tasks to generate a data index and processing the data in accordance with the data index to produce indexed data. The method then continues by determining distribution criteria for the data based on the data index and distributing the data and the data index to a set of distributed storage units in accordance with the distribution criteria, Finally, the method establishes criteria for analyzing found data of the data in the storage network.

Storing, processing and analyzing large volumes of data in a storage network

A processing system of a storage network operates by: selecting a queue memory type of a plurality of memory types to store a data object, based on a size parameter associated with the data object; storing the data object in a queue memory device having the queue memory type, when the queue memory type is selected; selecting a main memory type of a plurality of memory types to store the data object, when the queue memory type is not selected; and storing the data object in a main memory device having the main memory type, when the queue memory type is not selected; wherein the data object is dispersed error encoded and stored as a plurality of encoded data slices.

Storing data objects in a storage network with multiple memory types

Methods and apparatus for identifying multiple resources of a storage network for data retrieval are disclosed. In various embodiments, a determination is made to retrieve an encoded data slice from the storage network. Based on configuration information for the storage network, one or more configurations are identified, including a current configuration. For the identified configurations, ranked scoring information relating to the encoded data slice is determined for a plurality of resources associated with the identified configurations. Based on the ranked scoring information, a resource is selected for each of the identified configurations. In addition, a retrieval likelihood level for the data slice is determined for each of the selected resources. Based on the likelihood levels for the selected resources, one or more of the selected resources are identified for encoded data slice retrieval and read slice requests are issued to the one or more selected resources.

Identifying multiple resources of a storage network for data retrieval

A method begins by identifying a plurality of encoded data slices requiring rebuilding. The method continues by determining an amount of memory required for rebuilding the plurality of encoded data slices and allocating memory in one or more storage units for the rebuilding the plurality of encoded data slices as reserve memory. The method continues by obtaining a plurality of rebuilt encoded data slices associated with the plurality of encoded data slices requiring rebuilding and storing the plurality of rebuilt encoded data slices in the reserve memory.

Memory allocation for block rebuilding in a storage network

A processing system is operable to encode data to produce a first set of data slices based on a value of a width parameter. The data is stored based on maintaining storage of the first set of data slices across a set of storage units of a storage pool. Storage of the first set of data slices is maintained in the set of storage units of the storage pool after addition of an additional set of storage units added to the storage pool. The value of the width parameter is increased to an increased value to produce an updated width parameter. The data is re-encoded in accordance with the updated width parameter to produce a second set of data slices. The data is re-stored based on maintaining storage of the second set of data slices across the expanded set of storage units of the storage pool.

Re-encoding data in a storage network based on addition of additional storage units

A storage network operates by: encoding a data segment into a set of encoded data slices, wherein a read threshold of encoded data slices of the set of encoded data slices is required to decode the data segment, wherein the read threshold has a value greater than one, and wherein the data segment has an associated security level; selecting a subset of the plurality of storage units based on the security level, wherein the subset includes at least the read threshold of storage units of the plurality of storage units, wherein each of the subset of the plurality of storage units has a connection security approach that corresponds to the security level; and communicating the set of encoded data slices to the subset of the plurality of storage units in accordance with the connection security approach associated with each of the subset of the plurality of storage units.

Secure storage network and methods for use therewith

A method for execution by a computing device of a storage network includes obtaining at least a 'T' number of encoded data slices of a set of encoded data slices, where a plurality of data chunks are all-or-nothing encoded in accordance with distributed data storage parameters to produce the set of encoded data slices. The method further includes decoding a first section of the at least the “T” number of encoded data slices to recover a first data chunk of a plurality of data chunks. The method further includes decoding a second section of the at least the “T” number of encoded data slices to recover a second data chunk of the plurality of data chunks.

Recovering a data chunk from a portion of a set of encoded data slices

A data segment is encrypted to produce an encrypted data segment, and a data tag associated with the data segment is generated. The encrypted data segment is encoded to generate a set of encoded data slices. At least a read-threshold number of encoded data slices are required to reconstruct the encrypted data segment. A set of write slice requests, which includes the set of encoded data slices and the data tag, is transmitted to a DSN memory. A determination is made, based on the data tag, whether a first encoded data slice of the set of encoded data slices is a duplicate of a second encoded data slice already stored within the DSN memory. If it is a duplicate, rather of storing the first encoded data slice, a reference to a location of the second encoded data slice is stored.

Efficient storage of data in a dispersed storage network

Growing community of inventors

Chicago, IL, United States of America

Wesley B Leggette