Xerox Corporation

Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Douglas Brian Terry

Karin Petersen

John Ogden Lamping

James Douglass Thornton

Warren Keith Edwards

James P Dourish

Anthony George LaMarca

Michael P Salisbury

Marvin Michael Theimer

Michael J Spreitzer

Lev Novik

Akshat Vig

Irena Hudis

Alan J Demers

Brent B Welch

Mehul A Shah

Somasundaram Perianayagam

Vijayan Prabhakaran

Tate Andrew Certain

Vaibhav Jain

Yunxin Wu

Benjamin Albert Sowell

Alexander Richard Keyes

Rishabh Jain

Bohou Li

James Christopher Sorenson

Amit Gupta

Edward Pearce Wobber

Tomas Talius

Thomas Lee Rodeheffer

Ashish B Shah

Vivek Jawahir Jhaveri

Sanjay Anand

Derek William Richard Chen-Becker

David A Nichols

Venugopalan Ramasubramanian

Nathan Pellegrom Riley

Stefano Stefani

Roy Want

William Noah Schilit

David Goldberg

Craig Wesley Howard

Thomas Bradley Scholl

Marcos Kawazoe Aguilera

Benjamin Warren Mercier

Michael Ray Clark

Mark D Weiser

Richard J Goldstein

Daniel C Swinehart

Shaoyu Zhou

Ming-Chuan Wu

Almero Gouws

Richard Shawn Bice

Arijit Choudhury

Ramakrishna Rao Kotla

Ramanathan Padinjarel Somanathan

Lewis Bruck

Mahesh Balakrishnan

Sanjay Shanthakumar

Rashmi Krishnaiah Setty

Ray Manpreet Singh Matharu

Abhijeet More

Sudhir Konduru

Murtaza Doctor

Sachet Saurabh

Vinay Singh

Krishnaditya Kandregula

Okechukwu C Echeruo

Vikas Malik

Tushar Poddar

Mukesh Punhani

Timothy Matthew Lewis

Hina Naseer

Karthikeyani Palanisami

Michael F Salisbury

Karin Peterson

Brian M Oki

Anthony G Lamarca

Michael Sklar Chess

Grayson Osburn Vigeant Clendenon

Frank Benkstein

Abhijit Uday Pujare

Hussam Abu-Libdeh

Meg Walraed-Sullivan

Jeremie E Sartor

Jin He

Mark Scurrell

Asmita Barve

Sanket Vishram Chandorkar

Vinay Agrawal

A database table may be synchronously replicated across regions of a provider network. When a write request to an item of the table is received at one region, a local version of the item may be obtained as well as any additional versions of the item corresponding to inflight writes that target the item from a multi-region replication append-only log. An updated version of the item is then generated based on the local version of the item and the additional versions, which is appended to the multi-region replication append-only log. A conditional write is performed to update the local version of the item with the updated version of the item. If successful, a response indicates that the write request was successful.

Synchronous cross-region replication for a multi-writer database

Respective delegation records indicating that a first access controller and a second access controller have been authorized to grant access to a respective set of cells of a table of a storage management service are stored. In response to receiving an access request of a data accessor, permission records indicating that the access controllers have granted permissions to the data accessor to respective subsets of the table's cells are identified. A collection of cells of the table to which the data accessor has been granted access permission is identified using the permission records, and a response to the access request is generated using the collection.

Delegated fine-grained access control for data lakes

At a lock-less data management service, a multi-phase commit of a transaction is performed. The multi-phase commit includes a read set validation phase (in which a first set of timestamp-based conditions is used to determine whether the transaction has a read-write conflict), a pre-commit timestamps update phase (in which respective pre-commit timestamps associated with data items of the transaction's write set are set to a proposed commit time after verifying that the proposed commit time satisfies a second set of timestamp-based conditions), and a write initialization phase (in which respective new versions of individual data items of the write set are stored, without storing data item values indicated in the write set).

Lock-free timestamp ordering for distributed transactions

A protocol for implementing ACID transactions that provides snapshot isolation in a distributed setting that does not require synchronized clocks is described. The protocol ensures at commit time that transactions touching common objects do not commit out of order. The protocol can be used in the context of a distributed data lake built on an object store in which clients can transactionally add or remove objects from logical tables.

Transactional protocol for snapshot isolation without synchronized clocks

Techniques for using tree data structures to maintain a transactionally consistent set with support for time-travel queries are described. When a transaction commits, a new version of the tree data structure is created using a copy-on-write based method such that the tree shares internal nodes with previous trees to save space. This approach may be used in the implementation of a transactional data catalog in which the files that make up a table are stored in a transactional set.

Tree-based approach for transactionally consistent version sets

Techniques for implementing systems using transactional version sets are described. Transactional version sets or t-sets include a collection of elements, each having a collection of metadata. A t-set is transactional in that a sequence of updates to one or more t-sets are made within an atomic transaction. A t-set is versioned since each committed transaction that updates it produces a new timestamped version that can be accessed via time-travel queries.

Transactional version sets

A distributed database system maintains data for a logical table by storing, on a plurality of storage nodes, a collection of key-item pairs. An operation to delete a range of these items is processed by first storing, on each of the plurality of storage nodes, a tombstone record indicative of the items to be deleted. The tombstones are committed once each of the plurality of nodes has stored the tombstone. Items within the range are treated as deleted. A cleanup operation deletes items within the range and updates the tombstone.

Range deletes on a distributed database

Transaction requests may be ordered in a distributed database according to an independently assigned sequence. Different distributed system nodes, such as a transaction coordinator and a storage node may independently assign sequence numbers to requests to access a distributed database. A storage node may receive the request from a transaction coordinator with an assigned sequence number and another request to which the storage node may assign a sequence number. The storage node can then order performance of the requests based on the sequence numbers.

Ordering transaction requests in a distributed database according to an independently assigned sequence

A data delivery service of a service provider may receive respective job specifications for different data transfer jobs between computing infrastructure collections (e.g., data centers). A job specification for a data transfer job may include an amount of data to be transferred for the data transfer job, one or more destinations of data transfers for the data transfer job, and/or one or more flexibility parameters for successful transfer of the data for the data transfer job (e.g., a deadline to transfer the data, available data delivery techniques). The data delivery service may determine a schedule for performing different data transfer jobs between two or more infrastructures based on an analysis of the amount of data to be transferred for each job, the destinations of the data transfer for each job, the flexibility parameters for each job (e.g., included in the respective job specifications), and the connectivity between computing infrastructure collections.

Flexible scheduling of data transfers between computing infrastructure collections for efficient resource utilization

A geographically distributed system using a hybrid, multi-master design is disclosed. The system may use global tables, for which both reads and writes may be performed locally, but with a single region serving as a master region. All strongly consistent reads and writes may be sent to the master region, whereas eventually consistent reads and writes may be performed in any region with a replica. The master region may not be statically assigned but may move from region to region. The master region may be assigned manually by client preference or may be assigned and updated automatically based on the time of day or workload characteristics.

Multi-region, multi-master replication of database tables

Updates performed as part of transaction requests to a data store may be included in a stream of updates. Updates to items in the data store that are included in transactions determined not to be committed to the data store may be excluded from the stream of updates. Records in the stream of updates may include an identifier for the transaction that included the update described by the record. The identifier for the transaction may be used to identify updates to other items in the data store that are included in the same transaction.

Streaming committed transaction updates to a data store

Disclosed are systems and methods for generating consistent backups. A central coordinator informs each node storing a partition of the time to perform a backup. At the designated time, each node blocks updates for a corresponding time interval measured by its local clock. Each node performs the backup operation according to its own local clock. Consistent backups may be generated in spite of clock skew between the local clocks as long as the time interval is at least as long as a maximum local clock skew (among the nodes performing the backup). In some systems the maximum local clock skew may be reduced, by a round-trip update latency from a client, for example.

Consistent backups based on local node clock

An application programming interface is provided that allows applications to assign multiple service-level agreements to their data transactions. The service-level agreements include latency bounds and consistency guarantees. The applications may assign utility values to each of the service-level agreements. A monitor component monitors the various replica nodes in a cloud storage system for latency and consistency, and when a transaction is received from an application, the monitor determines which of the replica nodes can likely fulfill the transaction in satisfaction of any of the service-level agreements. Where multiple service-level agreements can be satisfied, the replica node that can fulfill the transaction according to the service-level agreement with the greatest utility is selected. The application may be charged for the transaction based on the utility of the service-level agreement that was satisfied.

Consistency-based service-level agreements in cloud storage environments

A distributed system includes full and partial replicas of a set of data items that may be inserted, modified, or deleted by any replica. Replicas may occasionally synchronize with other arbitrarily chosen replicas to learn about updates. A replica's knowledge includes one or more knowledge fragments, where each fragment indicates a set of items. A type of knowledge fragment, called a star knowledge fragment, contains versions associated with all items in the system. Star knowledge fragments are compact because the set of items stored at a replica need not be explicitly listed. Once all replicas know of all updates in the system, partial and full replicas will have the same compact star knowledge fragment.

Accumulating star knowledge in replicated data protocol

A system is disclosed for synchronizing partially-replicated collections using the concept of item-set knowledge and move-out notifications. Move-out notifications are provided to allow removal of changed items from replicas when the change in the item takes that item outside of the interest set of the replicas.

Move-in/move-out notification for partial replica synchronization

Growing community of inventors

San Carlos, CA, United States of America

Douglas Brian Terry