International Business Machines Corporation

Pure Storage, Inc.

Cleversafe, Inc.

Other

Manish Motwani

Jason K Resch

Wesley B Leggette

Andrew D Baptist

Ilya Volvovski

Greg R Dhuse

Ravi V Khadiwala

Gary W Grube

S Christopher Gladwin

Ethan S Wozniak

Thomas Franklin Shirley, Jr

Timothy W Markison

Yogesh Ramesh Vedpathak

Praveen Viraraghavan

Michael Colin Storm

Brian Frederick Ober

Bart Richard Cilfone

Thomas Darrel Cocagne

Bruno Hennig Cabral

Zachary J Mark

Daniel J Scholl

Trevor J Vossberg

Harsha Hegde

Niall John McShane

Scott Michael Horan

John Quigley

Joseph M Kaczmarek

Asimuddin Kazi

Dustin M Hendrickson

Rohan P Shah

Adam Michael Gray

Vance T Thornton

Jamie Bellanca

Renars W Narubin

Jordan Harrison Williams

Mingyu Li

Kumar Abhijeet

Thomas Dubucq

Matthew Michael England

Srinivas Palthepu

Ying Zhao Guo

Jesse Louis Young

Michael J Niedbala

James L Lester

Anthony J Baldocchi

Sarah Toledano

Jeremy S Jarczyk

Patrick Aaron Tamborski

Andrew George Peake

Franco Vincent Borich

Ram S Narayanan

Sanjaya Kumar

Shaorong Liu

Randy D Pfeifer

Bart Richard Clifone

Benjamin Lee Martin

Jerri Lynn Foster

Shikha Shree

Mark Duane Seaborn

Sebastien Vas

Alan M Frazier

Steven Mark Hoffman

Tyler K Reid

Lynn Foster

David A Bukiet

Ilya Volvolvski

Ryan Joseph Kuester

Lakshmi Krishna Priya Kala

Christopher S Gladwin

Jr Thomas Franklin Shirley

Methods and apparatus for redirecting data access requests in a storage network are disclosed. In an embodiment, a computing device associated with the storage network performs a method that includes identifying a (second) storage pool of the storage network to replace a first storage pool of the storage network. The method includes issuing a migration message to at least one of the first storage pool or the second storage pool. The migration message initiates a replication of data from the first storage pool to the second storage pool. The method further includes receiving a write data request for the first storage pool prior to completion of the replication of the data, and forwarding the write data request to the second storage pool for further processing.

Redirecting a data access request in a storage network

A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Storing encoded data slices in primary storage slots

Methods and apparatus for identifying multiple resources of a storage network for data retrieval are disclosed. In various embodiments, a determination is made to retrieve an encoded data slice from the storage network. Based on configuration information for the storage network, one or more configurations are identified, including a current configuration. For the identified configurations, ranked scoring information relating to the encoded data slice is determined for a plurality of resources associated with the identified configurations. Based on the ranked scoring information, a resource is selected for each of the identified configurations. In addition, a retrieval likelihood level for the data slice is determined for each of the selected resources. Based on the likelihood levels for the selected resources, one or more of the selected resources are identified for encoded data slice retrieval and read slice requests are issued to the one or more selected resources.

Identifying multiple resources of a storage network for data retrieval

A data segment is encrypted to produce an encrypted data segment, and a data tag associated with the data segment is generated. The encrypted data segment is encoded to generate a set of encoded data slices. At least a read-threshold number of encoded data slices are required to reconstruct the encrypted data segment. A set of write slice requests, which includes the set of encoded data slices and the data tag, is transmitted to a DSN memory. A determination is made, based on the data tag, whether a first encoded data slice of the set of encoded data slices is a duplicate of a second encoded data slice already stored within the DSN memory. If it is a duplicate, rather of storing the first encoded data slice, a reference to a location of the second encoded data slice is stored.

Efficient storage of data in a dispersed storage network

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data based on the at least one rebuilt encoded data slice and the first subset.

Rebuilding missing data in a storage network via locally decodable redundancy data

A computing device includes an interface configured to interface and communicate with a storage network, a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice associated with a data object, determines whether the encoded data slice is stored in the first memory and in response to a determination that the encoded data slice is not stored in the first memory, issues another data access request for the encoded data slice to a second memory, where the first memory includes access characteristics that are faster than the second memory. When a data access response including the encoded data slice is received from the second memory, a response including the encoded data slice is transmitted.

Utilizing memories of different operational speeds in a vast storage network

A method includes determining an information dispersal algorithm width number and determining a number of primary storage slots from a number of storage slots associated with a set of storage units deployed across multiple sites, where the number of primary storage slots is equal to or greater than the information dispersal algorithm width number. The method further includes determining a mapping of primary storage slots to storage units. The method further includes sending configuration information to the set of storage units that includes the mapping. The method further includes storing a set of encoded data slices in the primary storage slots in accordance with the configuration information, where a data segment is error encoded into the set of encoded data slices in accordance with the information dispersal algorithm width number and a decode threshold number, which is a number of encoded data slices are needed to reconstruct the data segment.

Mapping storage slots to a set of storage units

Methods and apparatus for proxying data access requests in a storage network are disclosed. In various embodiments, a computing device associated with a first storage location of the storage network performs a method that includes receiving a portion of data that is in the process of being replicated from a second storage location of the storage network. The computing device facilitates storage of the portion of data in the first storage location of the storage network. In response to receiving a read request for the portion of data during the data replication process, the computing device determines whether the portion of data is available from memory of the first storage location. In response to determining that the portion of data is available from memory of the first storage location, the computing device issues a read response that includes the portion of data.

Proxying a data access request in a storage network

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device is operable to receive a memory access request for a data object stored within the DSN, determine a realm for the memory access request, determine an authorization service for the realm and generate an authorization request for the memory access request. The computing device is further operable to transmit the authorization request to an authorization service, receive an authorization request response from the authorization service, determine whether the memory access request is authorized and process the memory access request.

Access authentication in a dispersed storage network

A method for execution by a storage unit in a dispersed storage network (DSN) includes selecting a storage zone of a memory device of the storage unit based on zone allocation parameters, and designating the selected storage zone as open for writes. A data slice is received via a network for storage. The data slice is written sequentially at a memory location of the one of storage zone based on determining that the storage zone is designated as open for writes. A pointer corresponding to the data slice that indicates the storage zone and the memory location is generated. A read request is received via the network from a requesting entity that indicates the data slice. The data slice is retrieved from the memory device based on the pointer, and is transmitted to the requesting entity.

Storing data sequentially in zones in a dispersed storage network

A method includes determining virtual memory use of a vault based on virtual memory use of one dispersed storage (DS) unit of a set of DS units that services the vault within the DSN, where data is dispersed storage error encoded in accordance with dispersed data storage parameters to produce a plurality of sets of error coded (EC) data slices that are stored in the vault. The method continues by comparing the virtual memory use of the vault to one or more vault thresholds. When the virtual memory use of the vault is less than or equal to a first vault threshold of the one or more vault thresholds, the method continues by instructing a memory state of the vault to be in a normal state, wherein when the vault is in the normal state, a write mode is activated allowing write requests of EC data slices to the vault.

Dispersed storage network file system directory

A method for execution in a storage network includes receiving a request pertaining to a data object, where the request is related to a specified contiguous data range within the data object and the data object is dispersed error encoded to produce a plurality of groups of encoded data slices. The method continues by converting the request into a plurality of partial requests, where each partial request of the plurality of partial requests includes at least one partial task and transmitting each partial request of the plurality of partial requests to a storage unit of a set of storage units. The method then continues by determining when a partial request response has been received for each partial request of the plurality of partial requests and when a partial request response has been received for each partial request of the plurality of partial requests, transmitting an indication that the request pertaining to the data object has been successfully executed.

Processing requests for a data range within a data object in a distributed storage system

Methods and apparatus for use in a storage network operate by: storing, in a first storage unit of a first set of storage units of the storage network, a first encoded data slice corresponding to at least one data object; assigning one or more additional storage units to the storage network to form a second set of storage units, the second set of storage units including the one or more additional storage units; migrating the first encoded data slice from the first storage unit to at least one of the one or more additional storage units of the second set of storage units; and reallocating a mapping of the first encoded data slice from the first storage unit to the at least one of the one or more additional storage units of the second set of storage units.

Migrating data in a vast storage network

A method begins by a processing module of a storage network analyzing storage network memory for a level of usability and based on the analyzing, selecting alternative memory available for receipt of encoded data slices stored in current memory, where a data object is segmented into a plurality of data segments and a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices. The method continues with the processing module determining whether to move encoded data slices from current memory to alternative memory and based on a determination to move slices, allocating alternative memory. Finally, the processing module moves at least some encoded data slices from a current memory to alternate memory and updates a memory assignment mechanism for the at least some encoded data slices.

Parity in a vast storage system using alternate memory

A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of slices that is required to decode the data segment; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices, wherein the first subset of encoded data slices is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein the number of encoded data slices in the first subset of the encoded data slices is less than the read threshold number; generating, via the at least one processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data generated from a second subset of the set of encoded data slices that includes the at least one missing encoded data slice; and recovering, via the at least one processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.

Recovering missing data in a storage network via locally decodable redundancy data

Growing community of inventors

Chicago, IL, United States of America

Manish Motwani