Microsoft Technology Licensing, LLC

Malay Kumar Bag

Alekh Jindal

Hiren Shantilal Patel

Shi Qiao

Carlo Aldo Curino

Rathijit Sen

Zhicheng Yin

Yiwen Zhu

Jieming Di

A serverless query processing system receives a query and determines whether the query is a recurring query or a non-recurring query. The system may predict, in response to determining that the query is the recurring query, a peak resource requirement during an execution of the query. The system may compute, in response to determining that the query is the non-recurring query, a tight resource requirement corresponding to an amount of resources that satisfy a performance requirement over the execution of the query, where the tight resource requirement is less than the peak resource requirement. The system allocates resources to the query based on an applicable one of the peak resource requirement or the tight resource requirement. The system then starts the execution of the query using the resources.

Resource optimization for serverless query processing

Techniques are described herein that are capable of selecting checkpoints of a database job. For instance, at compile time, temporal indicators associated with the query plans of the database job are determined. Each temporal indicator indicates first and second subsets of stages of the respective query plan. Values of attributes of each stage in at least each first subset are predicted using a machine learning technique. At the compile time, candidate stage(s) for each query plan are identified based on the respective candidate stage being a child of stage(s) in the corresponding second subset or not being a child of another stage in the respective query plan. The candidate stage(s) for each query plan are selectively chosen as respective checkpoint(s) based on whether the values of the attributes of each stage in at least the first subset of the stages of the respective query plan satisfy one or more criteria.

Data-driven checkpoint selector

Described herein is a system and method for detecting and reusing overlapping computations. Overlapping subgraphs of the query are determined using a normalized signature for a particular subgraph that identifies a particular subgraph across recurring instances of data. A normalized signature for each overlapping subgraph for the determined overlapping subgraphs of the query is provided. For each overlapping subgraph determined to be materialized: whether or not the particular subgraph has been materialized is determined using a precise signature corresponding to a normalized signature of the particular overlapping subgraph. The precise signature identifies a particular subgraph corresponding to the normalized signature within a particular recurring instance of data. When the particular subgraph has not been materialized, the subgraph is materialized and used to respond to the query. When the particular subgraph has been materialized, the materialized subgraph is used to respond to the query.

Computation reuse in analytics job service

Growing community of inventors

Kirkland, WA, United States of America

Malay Kumar Bag