Hewlett-packard Development Company, L.P.

Hewlett Packard Enterprise Development LP

Alistair Veitch

Arif Abdulhusein Merchant

John Wilkes

Minwen Ji

Svend Frolund

Susan Spence

Kimberly Keeton

James S Jackson

Yasushi Saito

James Michael Reuter

Charles B Morrey, Iii

Craig A Soules

Yasusuhi Saito

Marcos Kawazoe Aguilera

Mark David Lillibridge

Eric A Anderson

Parthasarathy Ranganathan

Ludmila Cherkasova

Christos Karamanolis

Douglas L Voigt

Mehul A Shah

Jichuan Chang

Mustafa Uysal

Ram Swaminathan

Vinay K Deolalikar

Richard Andrew Golding

Eric T Anderson

Sheng Li

Kevin T Lim

Guillermo Alvarez

Jishen Zhao

Craig A Souies

Craig A N Soules

Michael Hobbs

Ted Romer

A technique includes acquiring a plurality of write requests from at least one memory controller and logging information associated with the plurality of write requests in persistent storage. The technique includes applying the plurality of write requests atomically as a group to persistent storage.

Atomically committing write requests

A set of jobs to be scheduled is identified () in a system including a processing pipeline having plural processing stages that apply corresponding different processing to a data update to allow the data update to be stored. The set of jobs is based on one or both of the data update and a query that is to access data in the system. The set of jobs is scheduled () by assigning resources to perform the set of jobs, where assigning the resources is subject to at least one constraint selected from at least one constraint associated with the data update and at least one constraint associated with the query.

Resource assignment for jobs in a system having a processing pipeline that satisfies a data freshness query constraint

An embodiment of a method of restoring backup data comprises first and second steps that preferably are executed at least partially in parallel. The first step restores an incremental backup. The second step selectively restores a full backup. The selective restoration of the full backup disregards units of data that were restored from the incremental backup and restores a remainder of the full backup. In an alternative embodiment, the first step precedes the second step. In another alternative embodiment, the selective restoration of the full backup accesses a table that indicates the units of data contained in the incremental backup to determine which units of data to disregard in the full backup.

Method of restoring backup data

An embodiment of a method of writing data begins with a first step of generating a timestamp. A second step issues a query that includes the timestamp to each of a plurality of primary storage devices. The method continues with a third step of receiving a query reply from at least a quorum of the primary storage devices. The query replies indicate that the timestamp is later than an existing timestamp for the data. In a fourth step, the data is mirrored to secondary storage after receiving the query reply from at least the quorum of the primary storage devices. Upon receiving a mirror completion message from the secondary storage, a fifth step issues a write message that includes at least a portion of the data and the timestamp to each of the primary storage devices.

Method of writing data

Embodiments include methods, apparatus, and systems for snapshots in distributed storage systems. One method of software execution includes using a version tree to determine what data blocks are shared between various storage nodes in the version tree in order to create a clone or a snapshot of a storage volume in a distributed storage system that uses quorum-based replication.

Snapshots in distributed storage systems

The present invention provides techniques for assignment and layout of redundant data in data storage system. In one aspect, the data storage system stores a number M of replicas of the data. Nodes that have sufficient resources available to accommodate a requirement of data to be assigned to the system are identified. When the number of nodes is greater than M, the data is assigned to M randomly selected nodes from among those identified. The data to be assigned may include a group of data segments and when the number of nodes is less than M, the group is divided to form a group of data segments having a reduced requirement. Nodes are then identified that have sufficient resources available to accommodate the reduced requirement. In other aspects, techniques are providing for adding a new storage device node to a data storage system having a plurality of existing storage device nodes and for removing data from a storage device node in such a data storage system.

Redundant data assignment in a data storage system

To identify similar files in an environment having multiple client computers, a first client computer receives, from a coordinator computer, a request to find files located at the first client computer that are similar to at least one comparison file, wherein the request has also been sent to other client computers by the coordinator computer to request that the other client computers also find files that are similar to the at least one comparison file. In response to the request, the first client computer compares signatures of the files located at the first client computer with a signature of the at least one comparison file to identify at least a subset of the files located at the first client computer that are similar to the at least one comparison file according to a comparison metric. The first client computer sends, to the coordinator computer, a response relating to the comparing.

Identifying similar files in an environment having multiple client computers

A system has a processing pipeline with a plurality of processing stages, where each of the processing stages has one or plural processors, and where the processing stages are individually and independently scalable. A first of the processing stages of the processing pipeline provides a received date update into an update data structure, where the update data structure is accessible to process a query received by the system. One or more additional of the processing stages transforms the update data structure to allow for merging of the transformed update data structure into a database, where the transformed update data structure is accessible to process the query. Content of the transformed update data structure is stored into the database.

Storing update data using a processing pipeline

An embodiment of a method of cooperative caching for a distributed storage system begins with a step of requesting data from storage devices which hold the data. The method continues with a step of receiving any cached blocks and expected response times for providing non-cached blocks from the storage devices. The method concludes with a step of requesting a sufficient number of the non-cached blocks from one or more particular storage devices which provides an expectation of optimal performance.

Method of cooperative caching for distributed storage system

An embodiment of a method of writing data includes issuing write messages to a replica set of storage devices. Write confirmations are received from at least a majority of the storage devices. An embodiment of a method reading data includes issuing read messages to a replica set of storage devices. Read confirmations are received from at least a first majority of the storage devices. Read commit messages are issued to the storage devices. Commit confirmations are received from at least a second majority of the storage devices.

Methods of reading and writing data

An embodiment of a method of hashing an address space to a plurality of storage servers begins with a first step of dividing the address space by a number of the storage servers to form data segments. Each data segment comprises a base address. A second step assigns the data segments to the storage servers according to a sequence. The method continues with a third step of measuring a load on each of the storage servers. According to an embodiment, the method concludes with a fourth step of adjusting data shares assigned to the storage servers according to the sequence to approximately balances the loads on the storage servers while maintaining the base address for each data segment on an originally assigned storage server. According to another embodiment, the method periodically performs the third and fourth steps to maintain an approximately balanced load on the storage servers.

Method of hashing address space to storage servers

A unit of data is read from memory of a replicated cache. The replicated cache comprises a plurality of independent computing devices. Each independent computing device comprises a processor and a portion of the memory. Confirmations are received from at least a majority of the independent computing devices that a flush operation for the unit of data was initiated no later than a time indicated by the timestamp and that a more recent version of the unit of data has not been flushed. The unit of data is provided to storage.

Operating a replicated cache that includes receiving confirmation that a flush operation was initiated

Read, write and recovery operations for replicated data are provided. In one aspect, a system for redundant storage of data included a plurality of storage devices and a communication medium for interconnecting the storage devices. At least two of the storage devices are designated devices for storing a block of data. Each designated device has a version of the data and a first timestamp that is indicative of when the version of data was last updated and a second timestamp that is indicative of any pending update to the block of data. The read, write and recovery operations are performed to the data using the first and second timestamps to coordinate the operations among the designated devices.

Read, write, and recovery operations for replicated data

Data blocks are read from a distributed cache. The distributed cache comprises m replicated caches, each replicated cache including a plurality of independent computing devices. Each independent computing device of the replicated caches holds a replica of a particular one of the m data blocks in memory. The m data blocks and p parity blocks are stored across m plus p independent computing devices. Each of the m plus p independent computing devices stores a single block selected from the m data blocks and the p parity blocks.

Method of operating distributed storage system in which data is read from replicated caches and stored as erasure-coded data

A transactional shared memory system has a plurality of discrete application nodes; a plurality of discrete memory nodes; a network interconnecting the application nodes and the memory nodes, and a controller for directing transactions in a distributed system utilizing the shared memory. The memory nodes collectively provide an address space of shared memory that is provided to the application nodes via the network. The controller has instructions to transfer a batched transaction instruction set from an application node to at least one memory node. This instruction set includes one or more write, compare and read instruction subsets, and/or combinations thereof. At least one subset has a valid non null memory node identifier and memory address range. The memory node identifier may be indicated by the memory address range. The controller controls the memory node responsive to receipt of the batched transaction instruction set, to safeguard the associated memory address range during execution of the transaction instruction set. The batched transaction instruction set is collectively executed atomically. A notification instruction set may also be used to establish a notification, triggered upon a subsequent write event upon at least a portion of a specified address range.

Growing community of inventors

Mountain View, CA, United States of America

Alistair Veitch