Amazon Technologies, Inc.

Pivotal Software, Inc.

George Constantin Caragea

Michail Petropoulos

Carlos Garcia-Alvarado

Mohammad Foyzur Rahman

Gaurav Saxena

Naresh Chainani

Yannis Papakonstantinou

Gokul Soundararajan

Vuk Ercegovac

Sriram Krishnamurthy

Balakrishnan Narayanaswamy

Amr El-Helw

Mohamed F Soliman

Mohammad Rezaur Rahman

Venkatesh Raghavan

Chunbin Lin

Nikolaos Koulouris

Jiayuan Chen

Anurag Windlass Gupta

Andrew Edward Caldwell

Martin Grund

Raj Narayan Sett

Database systems may dynamically management concurrency levels for performing queries. A query may be received at a database system and a memory usage for the query may be predicted. A determination may be made as to whether available memory is enough to satisfy the predicted memory usage for the query. If the available memory is enough to satisfy the predicted memory usage for the query, then an increase in a concurrency level for performing queries at the database system may be made. The query may be allowed to execute concurrently with other queries according to the increased concurrency level.

Dynamic concurrency level management for database queries

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for precisely tracking memory usage in a multi-process computing environment. One of the methods includes implementing an instance of a memory usage tracker (MUT) in each process running in a node of a computer system. A MUT can maintain an account of memory usage for each of multiple logical owners running on a process on which the MUT is running. The MUT can determine an actual memory quota for each owner, and enforce the actual memory quota of the owner. Enforcing the actual memory quota of the owner can include receiving each memory allocation request, checking each allocation request and a current state of the account against the actual quota, approving or rejecting each allocation request, communicating the approval or rejection to an underlying memory manager, and updating the owner account for each approved allocation request.

Precisely tracking memory usage in multi-process computing environment

A database system may use a machine learning model creation system to create a machine learning model from data stored in the database system responsive to a request from a client. The database system may obtain an executable version of the machine learning model, based on an uncompiled hardware agnostic version of the machine learning model, according to the hardware configuration of one or more computing resources selected by the database system to perform requests to the database system that invoke the machine learning model to generate predictions.

Dynamic compilation of machine learning models based on hardware configurations

A database system may include a machine learning model which may be used to perform various data analytics for data stored in the database system. In response to a request to invoke the machine learning model to generate a prediction from data stored in the database system, the database system may perform one or more optimization operations, as part of a query plan, to prepare the data to make it suitable for use by the machine learning model.

Integrating query optimization with machine learning model prediction

Performance measures are predicted for queries to prioritize query performance at a database system. A trained machine learning model for the database system may be applied to a query to determine a predicted performance measure for the query. The predicted performance measure may be compared with other predicted performance measures for other waiting queries to determine a priority for executing the query.

Predicting query performance for prioritizing query execution

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for tracking memory usage. One of the methods includes implementing an instance of a memory usage tracker (MUT) in each process running in a node of a computer system. The MUT initiates a free process for a memory chunk. A current generation of the memory chunk is determined. The MUT determines whether the mapping element's generation precedes the current generation. If the mapping element's generation is old, the MUT treats the long-living rollover account as the owner of the memory chunk to be freed and if the current generation and the mapping element's generation match, then the MUT considers the owner associated with the mapping element as the owner of the chunk. The MUT then adjusts the balance of the determined memory account and/or sub-account and the balance of the mapping element by the freed memory amount.

Database queries may be performed using resources based on a determined size of the database query. Database query size may be dynamically determined for a database query when the query is received. The database query may be assigned to resources used for database queries of the determined size. In some embodiments, timeouts may be applied to reassign database queries to different resources if the performance of the database query exceeds a timeout threshold.

Using computing resources to perform database queries according to a dynamically determined query size

Cardinality values can be derived from an approximate quantile summary. An approximate quantile summary can be generated for a column of a database table at data ingestion, data update, upon request, in response to a query, and in various other scenarios. When a query is received that includes a predicate directed to the column of the approximate quantile summary, a cardinality value may be derived from the boundary values of one or more quantiles that include the predicate. The cardinality value may then be used to select a query plan. The query may be performed according to the selected query plan.

Deriving cardinality values from approximate quantile summaries

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating query plans for queries that include common table expressions. One of the methods includes receiving a query that includes a common table expression (CTE), wherein the CTE signifies that a system can compute a result for the query using a query plan that includes a CTE producer operator and a corresponding CTE consumer operator. A first query plan that includes a CTE producer operator and a corresponding CTE consumer operator and a second query plan that inlines an instance of the CTE consumer operator are generated. Respective costs are computed for the first query plan and the second query plan, and a result for the query is computed using the second query plan if the second query plan has a lower cost.

Query plan generation for common table expressions

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating query plans for queries that include common expressions. One of the methods includes receiving an initial query plan generated for a query that includes multiple common expressions; identifying two or more matching subtrees of the initial query plan; generating an alternative query plan that includes a CTE producer operator and having an instance of one of the matching subtrees as one or more descendant operators of the CTE producer operator, including replacing at least one instance of a matching subtree in the initial query plan with a CTE consumer operator corresponding to the CTE producer operator; computing respective costs for the initial query plan and the alternative query plan; selecting the alternative query plan based on the cost for the alternative query plan; and computing a result for the query using the alternative query plan instead of the initial query plan.

Growing community of inventors

San Mateo, CA, United States of America

George Constantin Caragea