Snowflake Inc.

Spyridon Triantafyllis

Matthias Carl Adams

Benoit Dageville

Martin Hentschel

Ashish Motivala

Simon Holm Jensen

Yi Fang

Yizhi Zhu

Lars Volker

Kevin Wang

In an embodiment, a data platform receives a query that includes a two-sided N dimensional interval join of first and second input relations, where N>1. The two-sided N dimensional interval join has an interval-join predicate that, in each of N dimensions, compares intervals determined from the first and second input relations. The data platform implements the interval join at least in part by identifying an intermediate relation that includes all combinations of a row from the first input relation and a row from the second input relation where, in each of the N dimensions, the intervals determined from the first and second input relations both overlap a common N dimensional domain region of an input domain of the first and second input relations. The data platform obtains and returns results of the query.

Implementing multidimensional two-sided interval joins on data platforms

Disclosed herein are systems and methods for performing a geospatial-function join using an implied interval join. In an embodiment, a database platform receives a query that includes a geospatial-function join, which applies a geospatial-function predicate to a first geography data object of a first relation and a second geography data object of a second relation. The database platform processes the first and second relations through an interval join that applies an interval-join predicate that is implied by the geospatial-function predicate. The database platform obtains query results at least in part by implementing a filter that applies the geospatial-function predicate to an output of the interval join, and outputs the query results.

Performing geospatial-function join using implied interval join

The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising information about data stored in a micro-partition of the table.

Multi-level data for database systems

In an embodiment, a data platform receives a query that includes a two-sided N dimensional interval join of first and second input relations. The data platform samples, with respect to each of one or more of the N dimensions, one or both of the first input relation and the second input relation with respect to an interval size of an interval determined from the input relation. The data platform demarcates the N dimensional input domain into non-overlapping N dimensional input-domain cells based on the sampling. The data platform implements the interval join using a query-execution plan that includes an equality join that is keyed on input-domain-cell identifiers of input-domain cells that at least partially overlap bounding polygons of the intervals determined from the first and second input relations. The equality join is followed in the query-execution plan by a filter that applies the interval-join predicate. The data platform obtains results of the query by executing the query-execution plan.

Implementing multidimensional two-sided interval joins using sampling-based input-domain demarcation

In an embodiment, a database platform receives query instructions for a query on a database, where the query instructions include instructions for a geospatial-function join that includes a geospatial-function predicate. The database platform generates a query-execution plan based on the query instructions, including replacing the geospatial-function join with one or more interval joins that each include one or more predicates implied by the geospatial-function predicate. The database platform inserts, into the query-execution plan above the one or more interval joins, a filter operator that applies the geospatial-function predicate. The database platform obtains results of the query at least in part by executing the query-execution plan, and returns the query results in response to the query.

Performing geospatial-function joins using interval joins

In an embodiment, a data platform implements a two-sided N dimensional interval join using an N dimensional band join followed by a filter that applies a predicate of the interval join. The data platform generates first and second modified relations from first and second input relations. Each modified relation includes a copy of each row from the corresponding input relation for each input-domain cell that overlaps, in each of N dimensions, a bounding polygon of intervals determined from the row of the corresponding input relation. The data platform inserts, in each row in each modified relation, an input-domain-cell identifier of the corresponding overlapping input-domain cell and uses a hash-equality join that receives the first and second modified relations and that is keyed on the input-domain-cell identifiers. The data platform obtains results of a query by executing a query-execution plan that includes the query-plan section.

Multidimensional two-sided interval joins on hash-equality-join infrastructure

Disclosed herein are systems and methods for pre-filter deduplication for multidimensional two-sided interval joins. In an embodiment, a data platform receives query instructions for a two-sided N dimensional interval join, where N is an integer greater than 1. The two-sided N dimensional interval join has an interval-join predicate that compares intervals determined from the input relations in each of N dimensions. The data platform implements the two-sided N dimensional interval join as a query-plan section that includes an N dimensional band join that is followed by a deduplication operator that is followed by a filter that applies the interval-join predicate. The N dimensional band join includes a hash join keyed to N dimensional domain cells overlapped at least in part by intervals determined from the input relations in each of the N dimensions. The deduplication operator removes duplicate rows from a potential-duplicates subset of the output of the N dimensional band join.

Pre-filter deduplication for multidimensional two-sided interval joins

Disclosed herein are systems and methods for point-based relation splitting in geospatial-function-implied interval joins. In an embodiment, a data platform receives a query that applies a geospatial-function predicate to first and second geography data objects from first and second relations. The second relation is divided into point and non-point subsets based on the second data geography object. The data platform routes the point subset along a first path that includes a one-sided interval join that applies, to the first relation and the point subset, an interval-join predicate implied by the geospatial-function predicate. The data platform routes the non-point subset along a second path that does not include the one-sided interval join. The data platform obtains query results at least in part with a filter that applies the geospatial-function predicate to outputs of the one-sided interval join and the second path, and outputs the query results.

Point-based relation splitting in geospatial-function-implied interval joins

Disclosed herein are systems and methods for implementing multidimensional two-sided interval joins on a distributed hash-based-equality-join infrastructure. In an embodiment, a data platform receives, for a query on a database, query instructions that include a two-sided N-dimensional interval join of a first input relation and a second input relation, where N is an integer greater than 1. The two-sided N-dimensional interval join has an interval-join predicate that, in each of N dimensions, compares an interval determined from the first input relation with an interval determined from the second input relation. The data platform generates a query-execution plan that implements the two-sided N-dimensional interval join as a query-plan section that includes an N-dimensional band join followed by a filter that applies the interval-join predicate to a band-join output of the N-dimensional band join. The data platform obtains results of the query at least in part by executing the query-execution plan.

Multidimensional two-sided interval joins on distributed hash-based-equality-join infrastructure

Disclosed herein are systems and methods for multidimensional and multi-relation sampling for implementing multidimensional two-sided interval joins. In an embodiment, a data platform receives query instructions for a two-sided N dimensional interval join, where N is an integer greater than 1. The two-sided N dimensional interval join has an interval-join predicate that compares intervals determined from the input relations in each of N dimensions. The data platform samples interval sizes in one or more input relations, and demarcates an N dimensional input domain based on the sampling. The data platform implements the two-sided N dimensional interval join using an N dimensional band join followed by a filter that applies the interval-join predicate. The N dimensional band join includes a hash join keyed to N dimensional domain cells overlapped at least in part by intervals in the input relations in each of the N dimensions.

Multidimensional and multi-relation sampling for implementing multidimensional two-sided interval joins

The subject technology receives first metadata corresponding to a set of micro-partitions. The subject technology stores a first data structure and a second data structure in storage as a first file and a second file, first data structure including the first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores third metadata for a table, the third metadata comprising: cumulative table metadata comprising global information about a plurality of micro-partitions of the table, the cumulative table metadata being stored in a metadata micro-partition associated with the table.

Storing metadata using data structures in database systems

The subject technology generates a first data structure including first metadata and a second data structure including second metadata, the first metadata corresponding to a set of micro-partitions, the second metadata for a grouping of the first metadata, the second data structure including information associating the second metadata to the first metadata. The subject technology stores the first data structure and the second data structure in storage as a first file and a second file. The subject technology stores third metadata for a table, the third metadata comprising: cumulative table metadata comprising global information about a plurality of micro-partitions of the table, the cumulative table metadata being stored in a metadata micro-partition associated with the table.

Generating data structures for storing database metadata

A database platform receives query instructions for a query on a database, where the query instructions include a geospatial-function join that includes a geospatial-function predicate. The database platform generates a query-execution plan by replacing the geospatial-function join with a single interval join that has an interval-join predicate that is implied by the geospatial-function predicate, and by also inserting a filter above the single interval join to apply the geospatial-function predicate. The database platform executes the query-execution plan, and returns query results in response to the query.

Performing geospatial-function join using single interval join

A database platform receives query instructions for a query on a database, where the query instructions include a geospatial-function join that applies a geospatial-function predicate to a first relation that includes a first join key and a second relation that includes at least one row in which a second join key is a point. The database platform generates a query-execution plan by replacing the geospatial-function join with a one-sided interval join of at least a first portion of the first relation and the at least one row of the second relation in which the second join key is a point, and by also inserting a filter above the one-sided interval join to apply the geospatial-function predicate. The database platform executes the query-execution plan, and returns query results in response to the query.

Growing community of inventors

San Mateo, CA, United States of America

Spyridon Triantafyllis