Snowflake Inc.

Sap Se

Ismail Oukid

Stefan Richter

Thierry Cruanes

Benoit Dageville

Mahmud Allahverdiyev

Norman May

Daniel Booss

Robert Lasch

Alejandro Salinger

Wolfgang Lehner

Max Heimel

Linnea Passing

Juliane K Waack

Ivan Schreter

Thomas Willhalm

Anisoara Nica

Peter Bumbulis

Selcuk Aya

Matthias Carl Adams

Bowei Chen

Carsten Binnig

Tiemo Bang

Peter Popov

Daniel Dos Santos Bossle

Adrien Lespinasse

Johan Lasperas

A method to perform an indexed geospatial search includes retrieving, by at least one hardware processor, a query specifying a geography data column and a constant geography object. A first plurality of hash functions of a first set of cells covering a surface associated with the geography data column is determined. A search index of a database including the geography data column is updated based on the first plurality of hash functions to obtain an updated search index. The query is executed on a reduced scan set of the database. The reduced scan set is based on the updated search index.

Indexed geospatial search

A method, a system, and a computer program product for decompressing data. One or more compressed blocks in a set of stored compressed blocks responsive to a request to access data in the set of stored compressed blocks are identified. String prefixes inside the identified compressed blocks are decompressed using front coding. String suffixes inside the identified compressed blocks are decompressed using a re-pair decompression. Uncompressed data is generated.

Advanced database decompression

Provided herein are systems and methods for indexed geospatial predicate search. An example method performed by at least one hardware processor includes decoding a query with a geospatial predicate. The geospatial predicate is configured between a geography data column and a constant geography object. The method further includes computing a first covering for a data value of a plurality of data values in the geography data column. The first covering includes a first set of cells in a hierarchical grid representation of a geography. The first set of cells represents a surface of the geography associated with the data value. A second covering is computed for the constant geography object. A determination is made on whether to prune at least one partition of a database organized into a set of partitions and including the geography data column based on a comparison between the first covering and the second covering.

Indexed geospatial predicate search

A query directed at a source table organized into a set of batch units is received. The query comprises a regular expression search pattern. The regular expression search pattern is converted to a pruning index predicate comprising a set of substring literals extracted from the regular expression search pattern. A set of N-grams is generated based on the set of substring literals extracted from the regular expression search pattern. A pruning index associated with the source table is accessed. The pruning index indexes distinct N-grams in each column of the source table. A subset of batch units to scan for data matching the query are identified based on the pruning index and the set of N-grams. The query is processed by scanning the subset of batch units.

Regular expression search query processing using pruning index

Techniques and solutions are described for implementing virtual domains. Computing resources in a computing environment are determined and assigned to one or more virtual domains. One or more data structures can be located in a given virtual domain. The computing resources assigned to a virtual domain can be dynamically reconfigured without affecting processes that submit tasks to be performed on data structures in the virtual domains. Tasks can be submitted to a dispatcher, which can determine the appropriate virtual domain for the task and forward the task to the determined virtual domain. Tasks are received by virtual domains and assigned to worker threads, which can access a data structure specified for a given task.

Data structure execution framework using virtual computing domains

A source table organized into a set of batch units is accessed. The source table comprises a column of data corresponding to a semi-structured data type. One or more indexing transformations for an object in the column are generated. The generating of the one or more indexing transformation includes converting the object to one or more stored data types. A pruning index is generated for the source table based in part on the one or more indexing transformations. The pruning index comprises a set of filters that index distinct values in each column of the source table, and each filter corresponds to a batch unit in the set of batch units. The pruning index is stored in a database with an association with the source table.

Scan set pruning for queries with predicates on semi-structured fields

A top K query directed at a table is received. The table is organized into multiple storage units. The top K query comprises a first clause to sort a result set in order and a second clause that specifies a limit on a number of results provided in response to the query. A table scan operator identifies a first set of rows from the table based on a scan set determined for the table and provides the first set of rows to a top K operator. The top K operator determines a current boundary based on the first set of rows and provides the current boundary to the table scan operator. The table scan operator prunes the scan set based on the current boundary and identifies a second set of rows from the table based on the pruning.

Pruning data based on state of top K operator

A pruning index is generated for a source table organized into a set of batch units. The source table comprises a column of semi-structured data. The pruning index comprises a set of filters that index distinct values in each column of the source table. Rather than reassembling an entire tree structure of the semi-structured data prior to indexing, the generating of the pruning index comprises traversing a reassembly hook object that represents a first portion of the semi-structured data that is subcolumnarized and traversing a residual object that represents a second portion of the semi-structured data that is not subcolumnarized. The reassembly hook object is traversed to identify values corresponding to the first portion of the semi-structured data and the residual object is traversed to identify values corresponding to the second portion. The pruning index is stored with an association with the source table.

Index generation using lazy reassembling of semi-structured data

A source table organized into a set of micro-partitions is accessed by a network-based data warehouse. A pruning index is generated based on the source table. The pruning index comprises a set of filters that indicate locations of distinct values in each column of the source table. A query directed at the source table is received at the network-based data warehouse. The query is processed using the pruning index. The processing of the query comprises pruning the set of micro-partitions of the source table to scan for data matching the query, the pruning of the plurality of micro-partitions comprising identifying, using the pruning index, a sub-set of micro-partitions to scan for the data matching the query.

Pruning index generation and enhancement

A table organized into a set of batch units is accessed. A set of N-grams are generated for a data value in the source table. The set of N-grams include a first N-gram of a first length and a second N-gram of a second length where the first N-gram corresponds to a prefix of the second N-gram. A set of fingerprints are generated for the data value based on the set of N-grams. The set of fingerprints include a first fingerprint generated based on the first N-gram and a second fingerprint generated based on the second N-gram and the first fingerprint. A pruning index that indexes distinct values in each column of the source table is generated based on the set of fingerprints and stored in a database with an association with the source table.

Processing queries using an index generated based on data segments

Indexed regular expression search with N-grams

A query directed at a table organized into a set of batch units is received. The query comprises a predicate for which values are unknown prior to runtime. A set of values for the predicate are determined based on the query. An index access plan is created based on the set of values. Based on the index access plan, the set of batch units are pruned using a pruning index associated with the table. The pruning index comprises a set of filters that index distinct values in each column of the table. The pruning of the set of batch units comprises identifying a subset of batch units to scan for data that satisfies the query. The subset of batch units of the table are scanned to identify data that satisfies the query.

Join query processing using pruning index

Lazy reassembling of semi-structured data

A query directed at a source table organized into a set of batch units is received. The query includes a pattern matching predicate that specifies a search pattern. A set of N-grams are generated based on the search pattern. A pruning index associated with the source table is accessed. The pruning index comprises a set of filters that index distinct N-grams in each column of the source table. The pruning index is used to identify a subset of batch units to scan for matching data based on the set of N-grams generated for the search pattern. The query is processed by scanning the subset of batch units.

Growing community of inventors

Berlin, Germany

Ismail Oukid