Hewlett-packard Development Company, L.p.

Hewlett-packard Company

Svend Frolund

Arif Abdulhusein Merchant

Alistair Veitch

Susan Spence

Yasusuhi Saito

Rachid Guerraoui

Fernando Pedone

Alistar Veitch

James S Jackson

James Michael Reuter

Kave Eshghi

Marcos Kawazoe Aguilera

Yasushi Saito

Jayaram Rajan Kasi

James Christopher Pruyne

Jari Koistinen

Ellis Y Chi

An embodiment of a method of writing data begins with a first step of generating a timestamp. A second step issues a query that includes the timestamp to each of a plurality of primary storage devices. The method continues with a third step of receiving a query reply from at least a quorum of the primary storage devices. The query replies indicate that the timestamp is later than an existing timestamp for the data. In a fourth step, the data is mirrored to secondary storage after receiving the query reply from at least the quorum of the primary storage devices. Upon receiving a mirror completion message from the secondary storage, a fifth step issues a write message that includes at least a portion of the data and the timestamp to each of the primary storage devices.

Method of writing data

The present invention provides techniques for assignment and layout of redundant data in data storage system. In one aspect, the data storage system stores a number M of replicas of the data. Nodes that have sufficient resources available to accommodate a requirement of data to be assigned to the system are identified. When the number of nodes is greater than M, the data is assigned to M randomly selected nodes from among those identified. The data to be assigned may include a group of data segments and when the number of nodes is less than M, the group is divided to form a group of data segments having a reduced requirement. Nodes are then identified that have sufficient resources available to accommodate the reduced requirement. In other aspects, techniques are providing for adding a new storage device node to a data storage system having a plurality of existing storage device nodes and for removing data from a storage device node in such a data storage system.

Redundant data assignment in a data storage system

An embodiment of a method of writing data includes issuing write messages to a replica set of storage devices. Write confirmations are received from at least a majority of the storage devices. An embodiment of a method reading data includes issuing read messages to a replica set of storage devices. Read confirmations are received from at least a first majority of the storage devices. Read commit messages are issued to the storage devices. Commit confirmations are received from at least a second majority of the storage devices.

Methods of reading and writing data

A unit of data is read from memory of a replicated cache. The replicated cache comprises a plurality of independent computing devices. Each independent computing device comprises a processor and a portion of the memory. Confirmations are received from at least a majority of the independent computing devices that a flush operation for the unit of data was initiated no later than a time indicated by the timestamp and that a more recent version of the unit of data has not been flushed. The unit of data is provided to storage.

Operating a replicated cache that includes receiving confirmation that a flush operation was initiated

Read, write and recovery operations for replicated data are provided. In one aspect, a system for redundant storage of data included a plurality of storage devices and a communication medium for interconnecting the storage devices. At least two of the storage devices are designated devices for storing a block of data. Each designated device has a version of the data and a first timestamp that is indicative of when the version of data was last updated and a second timestamp that is indicative of any pending update to the block of data. The read, write and recovery operations are performed to the data using the first and second timestamps to coordinate the operations among the designated devices.

Read, write, and recovery operations for replicated data

Data blocks are read from a distributed cache. The distributed cache comprises m replicated caches, each replicated cache including a plurality of independent computing devices. Each independent computing device of the replicated caches holds a replica of a particular one of the m data blocks in memory. The m data blocks and p parity blocks are stored across m plus p independent computing devices. Each of the m plus p independent computing devices stores a single block selected from the m data blocks and the p parity blocks.

Method of operating distributed storage system in which data is read from replicated caches and stored as erasure-coded data

An embodiment of a method of seeking consensus among computer processes begins with a first step of saving a new timestamp in a timestamp array for a particular process. The method continues with a second step of determining whether a most recent entry in a decision array includes a previously established consensus decision. In a third step, if the most recent entry does not include the previously established consensus decision, the method saves a proposed decision as a consensus decision. Otherwise, in a fourth step, the method saves the previously established consensus decision as the consensus decision. In a fifth step, if a most recent timestamp in the timestamp array continues to be the new timestamp, the method returns the consensus decision. Otherwise, in a sixth step, the method returns an abort indicator.

Method of seeking consensus among computer processes

A computer system and software are provided for customizing on-line computer services. An application is able to interact with a sub-system between which an interceptor system can be interposed. The interceptor system includes a proxy interacting with the application, a customization module interacting with the proxy and a dispatcher interacting with the customization module and interacting with the sub-system. A customization repository contains the customization module and interacts with a service and a client sharing the application and the sub-system. A control interacts with the customization repository and the service for causing customization of the service and the client by disposing the proxy to interact with the application and the dispatcher to interact with the sub-system.

System having customization modules to provide customizations

A method of reading data comprises receiving a request for a stripe of erasure coded data, sending read messages to at least a quorum of storage devices, and receiving at least the quorum of reply messages from the devices. The quorum of the reply messages includes at least a minimum number of stripe blocks needed to decode the data. The quorum meets a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks. A method of writing data comprises generating a timestamp, encoding the data, sending query messages including the timestamp to the storage devices, receiving query reply messages from each of at least the quorum of the devices, sending write messages to the devices, and receiving a write reply message from each of at least the quorum of the devices.

A method of reading data comprises sending read messages to storage devices holding the stripe and receiving at least a quorum of reply messages. The reply message from the storage device holding the data block includes the data block. The quorum meets a quorum condition of a number such that any two selections of the number of stripe blocks intersect in the minimum number of the stripe blocks needed to decode the stripe. A method of writing data comprises sending query messages to storage devices holding the stripe, receiving a query reply message from each of at least a first quorum of the storage devices, sending modify messages to the storage devices, and receiving a write reply message from each of at least a second quorum of the storage devices. The first and second quorums each meet the quorum condition.

A method of recovering a stripe of erasure coded data begins with sending query messages to storage devices. The method continues with receiving query reply messages from at least a first quorum of the storage devices. The query reply messages include a minimum number of the stripe blocks needed to decode the stripe. Following this, the stripe of erasure coded data is encoded. Next, a write message is sent to each of the storage devices, which include a timestamp and the stripe block destined for the storage device. The method concludes with receiving a write reply message from at least a second quorum of the storage devices indicating that the stripe block was successfully stored. The first and second quorums each meet a quorum condition of a number such that any two selections of the number of the stripe blocks intersect in the minimum number of the stripe blocks.

Method and recovery of data using erasure coded data from stripe blocks

A data service system includes web servers, each servicing any access request received by the data service system. Duplicate application servers are also provided, each processing any request directed from any one of the web servers. A session state information managing system is provided to allow different application servers to process requests belonging to a single session without requiring the requests to carry their entire session state information. The managing system includes a session state information manager that, when called by an application server in processing a request, (1) provides the session state information of the request to the application server, and (2) generate a state reference for a new session state information for the request after the application server has processed the request and generated the new session state information. The managing system also includes a store that stores all session state information.

Allowing requests of a session to be serviced by different servers in a multi-server data service system

Data structure and timestamp management techniques for redundant storage. A plurality of storage devices are interconnected by a communication medium. At least two of the storage devices are designated devices for storing a block of data. Each designated device stores a version of the data and a first timestamp that is indicative of when the version of data was last updated. A second timestamp is indicative of a pending update to the block of data. When the update to the block of data is completed at one of the designated devices, the device discards the second timestamp. A storage device acting as coordinator instructs the device to discard the second timestamp. The designated storage devices store a plurality of blocks of data and corresponding timestamps according to a data structure. At least some of the entries in the data structure correspond to a range of data blocks that share a common timestamp. Entries in the data structure are arranged such that the ranges do not overlap.

System for redundant storage of data

A distributed data center system protocol is provided which includes providing a client having a failure detector, a primary data center, and a backup data center operatively interconnected. A transaction operation is provided from the client to the primary database server in the primary data center and the backup database server for execution. If the client detects a failure or suspects a failure of the primary database server, the transaction operation is provided to the backup database server, which becomes the new primary database server. The database server executing the transaction operation returns the executed transaction operation to the client. If the primary data center suffers a disaster, the backup data center takes over control.

Distributed data center system protocol for continuity of service in the event of disaster failures

A hierarchical method is provided for fault tolerance in a distributed computer system. A plurality of data centers is provided having a plurality of objects in each of the plurality of data centers. A local sub-protocol is used for dissemination of messages within a data center in the plurality of data centers and the local sub-protocol is activated from another data center of the plurality of data centers in a single round-trip message in the absence of faults.

Growing community of inventors

Mountain View, CA, United States of America

Svend Frolund