Amazon Technologies, Inc.

Gaurav Saxena

Naresh Chainani

Mustafa Ozan Ozen

Mohammad Rezaur Rahman

Enrico Siragusa

Yannis Papakonstantinou

Vuk Ercegovac

Yongjun Wu

Timothy Andrew Rath

Ippokratis Pandis

Balakrishnan Narayanaswamy

Mingda Li

Lonnie J Princehouse

Andre Hernich

Sandeep Bhatia

George Constantin Caragea

Ravindra Ganti

Kent Haakan Karlsson

Nathaniel Stewart

Todd Jeffrey Green

Gaurav Gupta

Lei Li

Mengchu Cai

Dumanshu Goyal

Mairo Pedrini

Parminder Singh

Abhishek Sharma

Kiran Kumar Chinta

Patrick Joseph Lewis

Chunbin Lin

Atilim Cetin

James Claiborne Moore

Yanzhu Ji

Dmytro Bogatov

Mario Guerriero

Gokul Soundararajan

Omer Ahmed Zaki

Amarsingh Buckthasingh Winston

Punit Rajgaria

Murali Brahmadesam

Martin Grund

Karthik Gurumoorthy Subramanya Bharathy

Hemanth Satyanarayana

Sanuj Basu

Matthew Perry Abrams

Fabian Oliver Nagel

Euan McLeod

Benjamin Raphael Forman

Nabanita Maji

Davide Pagano

Panagiotis Parchas

Tim Kraska

Gopal Paliwal

Nirmesh Khandelwal

Eric Coll

Vadim Skipin

Zhengchun Liu

Joseph Michael Pavey

Bobby Castleberry

Stephen Han

Raj Narayan Sett

Nilesh Shahdadpuri

Sen Yue

Johannes Wust

Christoforos Svingos

Kaihui Zheng

Techniques for priority-based masking policy selection in a database environment are described. Masking policies are defined and attached to columns of relational data for particular users or roles. The attachment of a masking policy to a column includes a user-specified priority value. When multiple policies could apply to a particular query, the conflict can be easily resolved and understood by use of the priority values, for example, by selecting a candidate policy having a highest priority value.

Priority-based masking policy selection in a database environment

Techniques for dynamic database redaction using protected encryption secret material are described. A masking policy is defined that includes a reference to a secret material stored by a secrets manager service. The masking policy further identifies a pseudonymous redaction function that utilizes a cryptographic function requiring such a secret material. The secrets manager service is configured to grant access to the secret material by an entity of the database service that executes queries, such as a leader node of a cluster. For a particular query, the cluster obtains the secret material from the secrets manager service in a secure manner, uses the secret material for applying the cryptographic function to values for redaction purposes, and deletes any copies of secret material thereafter.

Dynamic database redaction using protected secret material

Methods for replicating transactional tables of a transactional database to an analytical database and maintaining updates to those transactional table representations are disclosed. Snapshots of the transactional tables are provided to the analytical database via a transport mechanism, such as a data storage service or a data streaming service, and stored at the analytical database. Then, checkpoints comprising portions of a change-data-capture log that has recorded transactional changes to the transactional tables of the transactional database are provided to the analytical database via the same or different transport mechanism and used to commit those transactional changes to the snapshot representations. The snapshot representations may be used to respond to incoming analytical queries in order to provide real-time querying results.

Hybrid transactional and analytical processing architecture for optimization of real-time analytical querying

A future workload may be predicted for a database system using analysis of queries submitted for execution. A feature vector for a query may be determined according to a query plan for the query. If the feature vector has not been previously seen, or has not been sufficiently seen, by the database system, a machine learning inference may be used to predict performance characteristics of the query, the machine learning system trained using previous feature vectors and performance characteristics of executed queries. If the feature vector has been sufficiently seen previously by the database system, a history of performance characteristics of previous queries with similar or the same feature vector may be used to predict the performance characteristics. The predictions may then be used to configure or reconfigure processing cluster(s) of the database system.

Query performance prediction using multiple experts

A materialized view created from one or more shared producer tables in separate producer clusters may be stored in a consumer cluster. The materialized view May undergo faster a refresh using an incremental refresh by obtaining snapshots of the producer table and incrementally refreshing based on deltas obtained by the snapshot. The materialized view may undergo various checks to determine whether the materialized view in the consumer resource is refreshable or unrefreshable based on schema compatibility. The materialized view may undergo further checks to determine whether the materialized view in the consumer resource is available or unavailable for incremental refresh based on detection of schema/data rearrangement operations on the producer table.

Update techniques for materialized views generated from tables shared from external databases

Methods, systems, and computer-readable media for continuous functions in a time-series database are disclosed. A plurality of data points of a time series are stored into one or more storage tiers of a time-series database. The plurality of data points comprise a plurality of discrete measurements at respective timestamps. Using one or more query processors of the time-series database, a query of the time series is initiated. The query indicates a time range. Using the one or more query processors, a continuous function is determined that represents a segment of the time series in the time range. The continuous function is determined based at least in part on the plurality of data points. An operation is performed using the continuous function as an input.

Continuous functions in a time-series database

A primary query engine may receive a query and determine whether the query is eligible for performance at a secondary query engine. If eligible, the primary query engine may evaluate the availability of the first query engine to perform the query. The first query engine may determine whether to assign the query to the primary query engine or to the secondary query according to availability evaluation. For queries assigned to the secondary query engine, the primary query engine may send a request to the secondary query engine to being processing of the query.

Dynamically assigning queries to secondary query processing resources

Stored procedures are generated to perform incremental updates to a materialized view for a database. When a request to create a materialized view is received, one or more internal tables are created from the database. A stored procedure is generated that when executed will update the materialized view of the database. The stored procedure may obtain changes to the database that were not included in the internal tables and update the internal tables to include the obtained changes. The stored procedure may be performed automatically and in response to requests.

Stored procedures for incremental updates to internal tables for materialized views

Database systems may dynamically management concurrency levels for performing queries. A query may be received at a database system and a memory usage for the query may be predicted. A determination may be made as to whether available memory is enough to satisfy the predicted memory usage for the query. If the available memory is enough to satisfy the predicted memory usage for the query, then an increase in a concurrency level for performing queries at the database system may be made. The query may be allowed to execute concurrently with other queries according to the increased concurrency level.

Dynamic concurrency level management for database queries

Context dependent execution time prediction may be applied to redirect queries to additional query processing resources. A query to a database may be received at a first query engine. A prediction model for executing queries at the first query engine may be applied to determine predicted query execution time for the first query engine. A prediction model for executing queries at a second query engine may also be applied to determine predicted query execution time for the second query engine. One of the query engines may be selected to perform the query based on a comparison of the predicted query execution times.

Context dependent execution time prediction for redirecting queries

Scaling of query processing resources for efficient utilization and performance is implemented for a database service. A query is received via a network endpoint associated with a database managed by a database service. Respective response times predicted for the query using different query processing configurations available to perform the query are determined. Those query processing configurations with response times that exceed a variability threshold determined for the query may be excluded. A remaining query processing configuration may then be selected to perform the query.

Scaling query processing resources for efficient utilization and performance

Atomic partition scheme updates to partition items may be implemented by a time series database. A time threshold may be assigned to partition scheme update so that the time threshold may be applied across a set of ingestion nodes that may apply the partition scheme update the same. A request to store an item with a timestamp less than the time threshold may be stored in one partition of the time series database, while the item may be stored in a different partition of the time series database if the item has timestamp greater than or equal to the time threshold.

Atomic partition scheme updates to store items in partitions of a time series database

Materialized views for a database system may be automatically refreshed according to performance benefits. Materialized views may be ordered according to determined performance benefits for the materialized views indicating the performance benefit obtained when a materialized view is used to perform a query at the database system. Materialized views may be selected for refresh operations according to the ordering based on a capacity of the database system to perform refresh operations.

Automatically refreshing materialized views according to performance benefit

Match rules for rewriting queries to use materialized views may be dynamically generated by a database system. A database system may generate rules that indicate whether a given query can use a materialized view and how to rewrite the given query to use the materialized view. A query may be received and the rules may be applied to the query to determine that the query can use the materialized view and to rewrite the query to use the materialized view. The rewritten query can then be executed.

Dynamic generation of match rules for rewriting queries to use materialized views

Performance measures are predicted for queries to prioritize query performance at a database system. A trained machine learning model for the database system may be applied to a query to determine a predicted performance measure for the query. The predicted performance measure may be compared with other predicted performance measures for other waiting queries to determine a priority for executing the query.

Predicting query performance for prioritizing query execution

Growing community of inventors

Seattle, WA, United States of America

Gaurav Saxena