Snowflake Inc.

Oracle International Corporation

Abdul Q Munir

Jeffrey Rosen

Thierry Cruanes

Jiaqi Yan

William H Waddington

Prasanna Rajaperumal

Benoit Dageville

Orestis Kostakis

Qiming Jiang

Prayag Chandran Nirmala

Khaled Yagoub

Jiansheng Huang

Cecilia Gervasio Grant

Shige Takeda

Karl Dias

Boyung Lee

Yi Pan

Joshua Klahr

Yuan Xie

Jinye Huo

Wenye Li

Jayanta Das

Hossein Ahmandi

A method includes parsing a query to determine a plurality of data processing operations associated with the query and including an AS OF JOIN operation between first time series data in a first table and second time series data in a second table. A query plan of the query is generated. The query plan includes a plurality of nodes corresponding to the plurality of data processing operations. At least one of the plurality of nodes corresponding to the AS OF JOIN operation is modified to generate a modified query plan of the query. The modifying is based on applying a UNION operation on at least a first portion of column data in the first table and the second table to obtain a combined table. Execution of the query by at least one of a plurality of computing nodes is scheduled based on the modified query plan.

Inexact timestamp range matching join for time series data (AS OF JOIN)

Resource provisioning systems and methods are described. In an embodiment, a system includes a plurality of shared storage devices collectively storing database data, an execution platform, and a compute service manager. The compute service manager is configured to determine a task to be executed in response to a trigger event and determine a query plan for executing the task, wherein the query plan comprises a plurality of discrete subtasks. The compute service manager is further configured to assign the plurality of discrete subtasks to one or more nodes of a plurality of nodes of the execution platform, determine whether execution of the task is complete, and in response to determining the execution of the task is complete, store a record in the plurality of shared storage devices indicating the task was completed.

Resource provisioning in database systems

A method, system, and apparatus for managing a persistent query result set are disclosed. A resource manager is disclosed that receives a query from a client over a computer network. A computer database server is disclosed that executes the query and generates a result set based on and in response to the received query. An execution platform is disclosed that divides the result set into a plurality of chunks, wherein each chunk comprises a portion of the generated result set. The resource manager adds metadata to each of the plurality of chunks. A first chunk of the plurality of chunks comprises metadata pointing to the plurality of chunks. The resource manager delivers at least the first chunk of the plurality of chunks to the client in response to the query. The execution platform offloads the remaining plurality of chunks to cloud storage in parallel.

Managing persistent database result sets

Disclosed herein are embodiments of systems and methods for selecting partitions for reclustering based on distribution of overlapping partitions. In an example, a database platform makes a determination to at least partially recluster a database table that includes data stored across a plurality of partitions. The database platform responsively selects a subset of the partitions. The selecting of the subset includes identifying a point on a domain of a clustering key that corresponds to a local maximum of overlapping partitions, and also includes selecting the subset from among a group of overlapping partitions. The group includes at least one partition that overlaps the identified point on the domain of the clustering key. Each partition in the selected subset is above a reduction goal of overlapping partitions. The database platform at least partially reclusters the selected subset based on the clustering key.

Selecting partitions for reclustering based on distribution of overlapping partitions

Disclosed herein are systems and methods for incremental reclustering of database tables based on local maxima of partition overlap. In an embodiment, a database platform makes a determination, based on one or more incremental-reclustering criteria, to incrementally recluster a database table, which has a clustering key and which is stored across a plurality of partitions. In response to making the determination, the database platform selects a subset of the partitions, and at least incrementally reclusters the selected subset. The selecting of the subset includes identifying a local maximum of a quantity of overlapping partitions in the plurality of partitions with respect to a domain of the clustering key of the table, where the overlapping partitions overlap with respect to the clustering key.

Incremental reclustering based on local maxima of partition overlap

The subject technology determines whether a table is sufficiently clustered. The subject technology in response to determining the table is not sufficiently clustered, selects one or more micro-partitions of the table to be reclustered. The subject technology constructs a data structure for the table. The subject technology extracts minimum and maximum endpoints for each micro-partition in the data structure. The subject technology sorts each of one or more peaks in the data structure based on height. The subject technology sorts overlapping micro-partitions based on width. The subject technology selects based on which micro-partitions are within the tallest peaks of the one or more peaks and further based on which of the overlapping micro-partitions have the widest widths.

Incremental reclustering of database tables using reclustering-count levels

Internal resource provisioning in database systems

The subject technology requests information related to usage history metadata from a metadata database. The subject technology receives the requested information from the metadata database, the requested information comprising information related to user demand. The subject technology predicts a size value indicating an amount of computing resources to request for executing a set of queries based on the usage history metadata. The subject technology determines, during a prefetch window of time within a first period of time, a current size of freepool of computing resources. The subject technology, in response to the current size of the freepool of computing resources being smaller than the predicted size value, sends a request for additional computing resources to include in the freepool of computing resources.

Providing resources using predicted size values

Managing freepool resources using predicted size values

Reclustering of database tables based on peaks and widths

The subject technology determines usage history metadata. The subject technology predicts a size value indicating an amount of computing resources to request for executing a set of queries based at least in part on the usage history metadata. The subject technology determines, during a prefetch window of time within a first period of time, a current size of a freepool of computing resources. The subject technology, in response to the current size of the freepool of computing resources being smaller than the predicted size value, sends a request for additional computing resources to include in the freepool of computing resources. The subject technology receives an indication that the request for additional computing resources was granted. The subject technology performs an operation to include the additional computing resources in the freepool of computing resources.

Adaptive freepool size prediction

Growing community of inventors

San Mateo, CA, United States of America

Abdul Q Munir