Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Ming-Chuan Wu

Sandeep Bhatia

Andrew Whitaker

Kanishka Chaturvedi

Maruthi Manohar Reddy Devarenti

VamsiKrishna Chaitanya Manchem

Nicolas Bruno

Surajit J Chaudhuri

Stefano Stefani

Tate Andrew Certain

Wei Xiao

Akshat Vig

Amit Gupta

Douglas Brian Terry

Alexandre Olegovich Verbitski

Srikanth Kandula

Richendra Khanna

Somasundaram Perianayagam

Jingren Zhou

Chiranjeeb Buragohain

Vaibhav Jain

Adam Douglas Morley

Rajaprabhu Thiruchi Loganathan

Dushyanth Narayanan

Knut Magne Risvik

Sharatkumar Nagesh Kuppahally

Miguel Oom Temudo De Castro

Alexander Richard Keyes

Alexander Shamis

Ravi Math

Aleksandar Dragojevic

Rishabh Jain

Lishi Jiang

Pravin Mittal

Murali M Krishna

Nathan Pellegrom Riley

Matthew Renzelmann

Abhijeet More

Dhruv Goel

Rajesh Iyer

Timothy Tan

Kimi Shah

Sameer Agarwal

Yuxi Bai

Hong Yang

Nilanjan Basu

Abhilash Reddy Koppula

Nitin Ahuja

Pushap Goyal

A non-relational database may be emulated using a relational database with a distributed data store. A request to access the non-relational database may be received at a frontend for a relational database engine that emulates an interface for a non-relational database engine. The request may be translated into the format for the relational database engine and performed by the relational database engine. The relational database engine may access data for the non-relational database at storage nodes that store the data in a relational table column of non-relational data type.

Emulating a non-relational database on a relational database with a distributed data store

Consistent queries to replicated data may be performed according to global clock values. Time values for updates performed at a data set may be identified according to a global clock. The updates may be replicated to a replicated portion of a data set. Queries directed to the replicated portion of the data set may be performed in response to determining that the time values for updates indicate that the replicated portion of the data set is consistent with a time value identified for the query according to the global clock.

Global clock values for consistent queries to replicated data

Replicated partitions of a database may be moved from one or more replica nodes to a different primary node. The replica nodes may maintain replicated partitions of one or more primary partitions of a primary node servicing access requests to the primary partitions. The replicated partitions may be copied to a different node in order to cause the processing of access requests directed to primary partitions to be serviced by the different node. Replicated partitions of a database may be moved from replica nodes to perform a split operation or a merger operation.

Moving database partitions from replica nodes

Transaction requests may be ordered in a distributed database according to an independently assigned sequence. Different distributed system nodes, such as a transaction coordinator and a storage node may independently assign sequence numbers to requests to access a distributed database. A storage node may receive the request from a transaction coordinator with an assigned sequence number and another request to which the storage node may assign a sequence number. The storage node can then order performance of the requests based on the sequence numbers.

Ordering transaction requests in a distributed database according to an independently assigned sequence

Processing and storage responsibility for a data set may be split according to separately stored shards of the data set. As one or more loads associated with shards of the dataset grow a re-sharding operation may be performed to reduce loading of particular shards and nodes that host the particular shards. A re-sharding operation may cause only a sub-set of as set of shards of the dataset to be split and only cause second portions of the split shards to be stored in additional computing nodes. In some embodiments, a number of shards to be included in the sub-set of shards to be split may be selected based on an overall number of shards in the set and a largest number in the Fibonacci sequence that is less than the overall number of shards in the set.

Data re-sharding

Multi-type attribute indexes may be implemented for document databases. When a request to perform a query at a document database is received that is directed toward in indexed attribute that has multiple data types for values stored in the multi-type attribute index, a predicate in the query may be replaced with a different predicate that is applicable to search the multi-type attribute index according to a sort order for the multiple data types stored in the at multi-type attribute index. A plan that includes the different predicate may be performed in order to provide a result of the query to a user.

Multi-type attribute index for a document database

A distributed graph database that enables scaling and efficient processing is described. The distributed graph database can, for example, scale up to petabytes of data to enable transactional processing of graph data with low latency and low processing overhead. The distributed graph database can include a cluster of devices and a remote direct memory access (RDMA)-based communication layer to perform low latency messaging between devices of the cluster of devices. Additionally, the distributed graph database can include a shared memory layer that provides one or more data structures, a transaction layer to facilitate query processing, and a graph database layer stored in computer-readable media and executed on a processor to implement a graph data model. In at least one example, the graph data model can be mapped to the one or more data structures.

Distributed graph database

Processing responsibility for a data set may be split according to separately stored partitions of the data set. A data set may be stored in different storage units that separately store different partitions of the data set. Mapping information may be maintained indicating the assignment of a node to process access requests to multiple partitions of a data set. A split event may be detected for the node, and one or more partitions assigned to the node may be reassigned to a different node. The storage units for the reassigned partitions may be made accessible to the different node. Then mapping information can be updated to indicate change in assignment from the original node to the different node so that the different node can begin processing access requests for the reassigned partitions.

Splitting processing responsibility for separately stored data partitions

The use of statistics collected during the parallel distributed execution of the tasks of a job may be used to optimize the performance of the task or similar recurring tasks. An execution plan for a job is initially generated, in which the execution plan includes tasks. Statistics regarding operations performed in the tasks are collected while the tasks are executed via parallel distributed execution. Another execution plan is then generated for another recurring job, in which the additional execution plan has at least one task in common with the execution plan for the job. The additional execution plan is subsequently optimized based at least on the statistics to produce an optimized execution plan.

Parallel computing execution plan optimization

Systems and methods that enable resolving deadlock cycles by victim selection based on cost and/or priority of strongly connected components in a static wait-for graph. As such, victims can be selected by iteratively estimating total deadlock resolution cost (based on cost of strongly connected components) for remaining deadlock participants, selecting the participant with the least estimated total cost as the next victim, and reducing the wait-for graph after removing the victim vertex. The victim selection algorithm can be a cost-based greedy algorithm, wherein during each victim selection iteration, a single optimal victim is selected based on estimated remaining cost.

Priority and cost based deadlock victim selection via static wait-for graph

A method of estimating cardinality of a join of tables using multi-column density values and additionally using coarser density values of a subset of the multi-column density attributes. In one embodiment, the subset of attributes for the coarser densities is a prefix of the set of multi-column density attributes. A number of tuples from each table that participate in the join may be estimated using densities of the subsets. The cardinality of the join can be estimated using the multi-column density for each table and the estimated number of tuples that participate in the join from each table.

Cardinality estimation of joins

Growing community of inventors

Bellevue, WA, United States of America

Ming-Chuan Wu