Google Inc.

Sebastian Kanthak

Alexander Lloyd

Brian Frank Cooper

Andrew Fikes

Sanjay Ghemawat

Yasushi Saito

Christopher Scott Taylor

Iulian Moraru

Mert Akdere

Jeffrey Adgate Dean

Wilson Cheng-Yi Hsieh

Michael James Boyer Epstein

Alexander Khesin

Chris Jorgen Taylor

Michal Piotr Szymaniak

Sean Quinlan

Clifford Arthur Frey

Christopher Cunningham Frost

Christopher Taylor

In a distributed database, a transaction is to be committed at a first coordinator server and one or more participant servers. The first coordinator server is configured to receive a notification that each participant server of the transaction is prepared at a respective prepared timestamp, the respective prepared timestamp being chosen within a time range for which the respective participant server obtained at least one lock. The first coordinator server computes the commit timestamp for the transaction equal or greater than each of the prepared timestamps, and restrict the commit timestamp such that a second coordinator server sharing at least one of the participant servers for one or more other transactions at a shared shard cannot select the same commit timestamp for any of the other transactions. The transaction is committed at the commit timestamp

Including transactional commit timestamps in the primary keys of relational databases

Systems and methods are provided for sending and receiving remote procedure calls (RPCs). Based on a message in a first RPC, a second set of RPCs are created by one or more computing devices, where each of the second set of RPCs includes a portion of the message in the first RPC. The message in the first RPC is replaced with a reference to each RPC of the second set of RPCs. The one or more computing devices send the first RPC with the references, and also send the second set of RPCs.

System and method for sending and receiving remote procedure calls

The technology relates to cosharding tables within a distributed storage system. A data table including one or more rows may be received. Each row in the data table may include an identifier key and pieces of data. Each piece of data in the data table may be indexed into individual rows of an index table, wherein each row in the index table includes data associated with the identifier key of the data table from which the piece of data in the respective row was indexed. The index table may be sharded into splits, wherein the sharding includes assigning each row of the index table into one of the splits based on the identifier key of the data table from which the piece of data in the respective row was indexed. The splits may be stored into two or more portions of the distributed storage system.

Cosharding and randomized cosharding

In a distributed system where a client's call to commit a transaction occurs outside the transaction's lock-hold interval, computation of timestamp information for the transaction is moved to a client library, while ensuring that no conflicting reads or writes are performed between a time of the computation and acquiring all locks for the transaction. The transaction is committed in phases, with each phase being initiated by the client library. Timestamp information is added to the locks to ensure that timestamps are generated during lock-hold intervals. An increased number of network messages is thereby overlapped with a commit wait period in which a write in a distributed database is delayed in time to ensure concurrency in the database.

Reducing commit wait in a distributed multiversion database by reading the clock earlier

Systems and methods are provided for efficiently sending large units of data in a network. A sender computing device splits the proposal into a plurality of fragments, and sends each of the plurality of fragments to a receiver computing device. The sender computing device receives an acknowledgement from the receiver computing device for each fragment received by the receiver computing device. If no acknowledgements for any fragments are received within a predetermined time period, it may be determined that an error occurred, in which case the sender may time out or take another action. The sender computing device further sends a message to the receiver computing device, wherein a payload of the message includes one or more references corresponding to the plurality of fragments sent.

System and method for sending proposals within a distributed state machine replication system

Methods and systems for a distributed transaction in a distributed database system are described. One example includes identifying a request to insert a split point in a source group comprising one or more tablet replicas, each tablet including at least a portion of data from a table in the distributed database system, and the split point splitting data in the source group into a first range and a second range different than the first range; in response to the request: sending a list of filenames in the first range of the source group to a first target group comprising one or more tablet replicas; and creating, at the first target group, a virtual copy of files represented by the list of filenames in the first range, the virtual copy making data of the files available, each using a new name, without duplicating the data of the files.

Splitting and moving ranges in a distributed system

A system, computer-readable storage medium storing at least one program, and a computer-implemented method for identifying a storage group in a distributed storage system into which data is to be stored is presented. A data structure including information relating to storage groups in a distributed storage system is maintained, where a respective entry in the data structure for a respective storage group includes placement metrics for the respective storage group. A request to identify a storage group into which data is to be stored is received from a computer system. The data structure is used to determine an identifier for a storage group whose placement metrics satisfy a selection criterion. The identifier for the storage group whose placement metrics satisfy the selection criterion is returned to the computer system.

Storing and moving data in a distributed storage system

A computer system issues a batch read operation to a tablet in a first replication group in a distributed database and obtains a most recent version of data items in the tablet that have a timestamp no great than a snapshot timestamp T. For each data item in the one tablet, the computer system determines whether the data item has a move-in timestamp less than or equal to the snapshot timestamp T, which is less than a move-out timestamp, and whether the data item has a creation timestamp less than the snapshot timestamp T, which is less than or equal to a deletion timestamp. If the determination is true, the computer system determines whether the move-out timestamp has an actual associated value and, if so, the computer system determines a second tablet in a second replication group in the database that includes the data item and issues the snapshot read operation to the second table in the second replication group to obtain a most-recent version of the data item that has a timestamp no greater than the snapshot timestamp T; otherwise, the computer system issues the snapshot read to the one tablet to obtain a most recent version of the data item that has a timestamp no greater than the snapshot timestamp T.

Efficient snapshot read of a database in a distributed storage system

A system, computer-readable storage medium storing at least one program, and a computer-implemented method for committing transactions on remote servers is presented. Commit requests are issued to remote servers in a set of remote servers to request that the remote servers in the set of remote servers agree to commit a transaction at a first designated future time. When responses from the remote servers in the set of remote servers are received before a first abort time and indicate that all remote servers in the set of remote servers have agreed to commit the transaction at the first designated future time, commit commands are issued to the remote servers in the set of remote servers instructing the remote servers to perform the transaction at the first designated future time.

System and method for committing transactions on remote servers

A distributed storage system is provided. The distributed storage system includes multiple front-end servers and zones for managing data for clients. Data within the distributed storage system is associated with a plurality of accounts and divided into a plurality of groups, each group including a plurality of splits, each split being associated with a respective account, and each group having multiple tablets and each tablet managed by a respective tablet server of the distributed storage system. Data associated with different accounts may be replicated within the distributed storage system using different data replication policies. There is no limit to the amount of data for an account by adding new splits to the distributed storage system. In response to a client request for a particular account's data, a front-end server communicates such request to a particular zone that has the client-requested data and returns the client-requested data to the requesting client.

Growing community of inventors

San Jose, CA, United States of America

Sebastian Kanthak