Amazon Technologies, Inc.

Bohou Li

Mehul A Shah

Benjamin Albert Sowell

Douglas Brian Terry

Vijayan Prabhakaran

Anurag Windlass Gupta

George Steven McPherson

Prajakta Datta Damle

Gopinath Duddi

Vinay Agrawal

Vikas Malik

Tushar Poddar

Mukesh Punhani

A protocol for implementing ACID transactions that provides snapshot isolation in a distributed setting that does not require synchronized clocks is described. The protocol ensures at commit time that transactions touching common objects do not commit out of order. The protocol can be used in the context of a distributed data lake built on an object store in which clients can transactionally add or remove objects from logical tables.

Transactional protocol for snapshot isolation without synchronized clocks

Data transformation workflows may be generated to transform data objects. A source data schema for a data object and a target data format or target data schema for a data object may be identified. A comparison of the source data schema and the target data format or schema may be made to determine what transformations can be performed to transform the data object into the target data format or schema. Code to execute the transformation operations may then be generated. The code may be stored for subsequent modification or execution.

Generating data transformation workflows

Techniques for using tree data structures to maintain a transactionally consistent set with support for time-travel queries are described. When a transaction commits, a new version of the tree data structure is created using a copy-on-write based method such that the tree shares internal nodes with previous trees to save space. This approach may be used in the implementation of a transactional data catalog in which the files that make up a table are stored in a transactional set.

Tree-based approach for transactionally consistent version sets

Techniques for implementing systems using transactional version sets are described. Transactional version sets or t-sets include a collection of elements, each having a collection of metadata. A t-set is transactional in that a sequence of updates to one or more t-sets are made within an atomic transaction. A t-set is versioned since each committed transaction that updates it produces a new timestamped version that can be accessed via time-travel queries.

Transactional version sets

Growing community of inventors

Palo Alto, CA, United States of America

Bohou Li