Snowflake Inc.

Bicheng Gao

Jiaqi Yan

Srinivas Vemuri

Nithin Mahesh

Nicola Dan Onose

Kavinder S Dhaliwal

Valeri Kim

Jiaqi Zhang

The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

Hidden database objects over external data

A source table can be provided by a provider account in a data system. A secure view of the source table is provided to one or more consumer accounts, the secure view limiting access to a subset of data in the source table. A plan to execute a command using the secure view may be generated, the plan including a secure view boundary on a subset of operations defining the secure view. The plan may be modified to move a first operation that was outside the secure view boundary to within the secure view boundary to generate a second plan to optimize performance in view of limitations or restrictions placed by the secure view.

Performance optimizations for secure objects evaluations

The subject technology receives, during a query compilation process, a query directed to a set of source tables. The subject technology performs, during the query compilation process, a modification of the query for adjusting at least one pruning operation. The subject technology determines, during a pruning process of a second query, the second query directed to a set of files in a database system and including a set of pruning operations on the set of files, whether to perform a pruning cutoff on the set of pruning operations, the pruning process performing a depth first search of a pruner tree structure, the set of files comprising a set of micro-partitions. The subject technology performs the pruning cutoff based on the determining, the pruning cutoff ceasing at least one pruning operation from the set of pruning operations.

Cutoffs for pruning of database queries

During a query compilation process, a query is received that is directed to a set of source tables, each source table from the set of source tables being organized into at least one micro-partition and the query including at least one pruning operation. During the query compilation process, a modification of the query is performed for adjusting the at least one pruning operation, the modification being based on a set of statistics collected for previous pruning operations on at least a portion of the set of source tables and a set of heuristics, the set of statistics indicating at least an amount of execution time for each previous query associated with each of the previous pruning operations. The query is compiled including the modification of the query. The compiled query is provided to an execution node of a database system for execution.

Automatic pruning cutoff in a database system

The subject technology receives, during a query compilation process, a query directed to a set of source tables, each source table from the set of source tables being organized into at least one micro-partition and the query including at least one pruning operation. The subject technology performs, during the query compilation process, a modification of the query for adjusting the at least one pruning operation, the modification being based at least in part on a set of statistics collected for previous pruning operations on at least a portion of the set of source tables and a set of heuristics. The subject technology compiles the query including the modification of the query. The subject technology provides the compiled query to an execution node of a database system for execution.

Growing community of inventors

San Mateo, CA, United States of America

Bicheng Gao