Snowflake Inc.

Oracle International Corporation

Snowflake Lnc.

Jiaqi Yan

Thierry Cruanes

Prasanna Rajaperumal

Allison Waingold Lee

Benoit Dageville

Igor Demura

William H Waddington

Varun Ganesh

Abdul Q Munir

Jeffrey Rosen

Libo Wang

Florian Andreas Funke

Shrainik Jain

Guangyan Hu

Qiuye Jin

Stratis Viglas

Yongsik Yoon

Nicola Dan Onose

Boyung Lee

Mohamed Zait

Ryan Michael Thomas Shelly

Bowei Chen

Bicheng Gao

Istvan Cseri

Subramanian Muralidhar

Marcin Zukowski

Dinesh Das

Sangyong Hwang

Srinivas Vemuri

Xuelai Cui

Jaeha Lee

Ganeshan Ramachandran Iyer

Li Yan

Yujie Li

Yucan Liu

Shiyu Qu

Vlad Bunescu

Xinglian Liu

Selcuk Aya

Muthunagappan Muthuraman

Prashant Gaharwar

Zerui Wei

Nithin Mahesh

David Schultz

Niloy Mukherjee

Polita Paulus

Nileema Shingte

Sunil Parrokkaren Chakkappen

Raja Suresh Krishna Balakrishnan

Yizhi Zhu

Jaeyun Noh

Marianne Shaw

Nirav Vyas

Wai Sing Wong

Yan Huang

Steven P Herbert

Marcia Feitel

James Malone

Kavinder S Dhaliwal

Vineet Garg

Leonid Baraznenok

Adrian Peter Neumann

Saiyang Gou

Wenhao Song

Juan R Loaiza

Tirthankar Lahiri

Vineet Marwah

Damien Carru

Shasank Kisan Chavan

Jesse Kamp

Aurosish Mishra

Kartik Kulkarni

Yi Fang

Brian Charles Forney

Nitish Jindal

Xinzhu Cai

Joshua Klahr

Louis Magarshack

Dennis Huo

Ronald Lee Ortloff

Weili Gu

Mohammad Saber Mirzaei

Donghe Kang

Alvin E Jou

Kevin Ye Jie Li

Andrew Xue

Dmitry A Lychagin

Mengran Wang

Prateek Gaur

Zhuo Peng

Valeri Kim

Jiaqi Zhang

Can Tuzla

Karan Chadha

Satyanarayana R Valluri

Harry Liu

Techniques for reducing query compilation time are described. Dynamic compilation behavior based on query properties can reduce time and computational costs. For example, the techniques can identify stages and rewrite rules to skip or modify in the compilation process based on, among other things, properties of the parse tree and the query plan.

Dynamic changes to compilation based on query properties

A data platform that implements memoizable functions for database objects. The data platform detects a first execution of a memoizable function and generates a first key based on metadata of one or more database objects operated on by the memoizable function and generates a first result for the memoizable function based on the one or more database objects. The data platform detects a second execution of the memoizable function and generates a second key based on the metadata of the one or more database objects operated on by the memoizable function. When the first key and the second key are equal, the data platform reuses the first result of the memoizable function. When the first key and second key do not match, the data platform generates a second result for the second execution of the memoizable function.

User defined function memoization

A method for selecting micro-partitions for a clustering operation includes: storing table data in a plurality of micro-partitions of a storage device, wherein each of the plurality of micro-partitions comprises a portion of the table data, wherein subsets of the plurality of micro-partitions are associated with a respective one of a plurality of expression property (EP) files, and wherein each of the plurality of EP files comprises an EP data region that represents the portions of the table data of the subset of the plurality of micro-partitions associated with the EP file; determining sub-ranges of the table data based on the EP data regions of the plurality of EP files; selecting a subset of the plurality of EP files for a clustering operation based on the sub-ranges of the table data; and performing the clustering operation on the micro-partitions associated with the subset of the EP files.

Micro-partition clustering based on expression property metadata

The subject technology receives a query, the query referencing a unified representation for structured type data and semi-structured type data, the unified representation being provided in storage and in memory during query processing, the unified representation comprising a set of structured type fields that include a set of semi-structured typed fields that enables type safety and enforcement for the set of structured type fields, and flexibility for the set of semi-structured typed fields in a same column, the unified representation in storage including type information for the semi-structured type data as part of the semi-structured type data, the unified representation being utilized for structured type data and semi-structured type data. The subject technology processes the query using the unified representation stored in the memory, the unified representation providing performance parity between structured type data and semi-structured type data.

Unified structured and semi-structured data types in database systems

The subject technology receives a query, the query including a statement for performing the query. The subject technology performs a lookup operation on a stored plan cache based on the query. The subject technology performs, in response to a cache match of the query to a stored query plan in the stored plan cache based on the lookup operation, a validation process of the stored query plan. The subject technology determines whether the stored query plan is valid based on the validation process. The subject technology performs, in response to determining that the stored query plan is valid, a program building process for the stored query plan to generate a final query plan. The subject technology sends the final query plan to an execution node for execution.

Optimized validation of cached query plans

The subject technology receives a query. The subject technology performs a lookup operation on a shadow cache. The subject technology performs a validation process on the particular query plan for determining a cache hit on the shadow cache. The subject technology performs a program building process to update the particular query plan with additional information for executing the particular query plan. The subject technology performs a compilation process to compile an updated program of the updated particular query plan. The subject technology determines that the updated particular query plan is cacheable based at least in part on the compilation process. The subject technology registers the updated particular query plan as a new entry in the shadow cache. The subject technology sends the updated program to an execution node for executing the updated particular query plan.

Query plan cache shadow modes

Systems, methods, and devices for sharing materialized views in multiple tenant database systems. A method includes defining a materialized view over a source table that is associated with a first account of a multiple tenant database. The method includes defining cross-account access rights to the materialized view to a second account such that that second account can read the materialized view without copying the materialized view. The method includes modifying the source table for the materialized view. The method includes identifying whether the materialized view is stale with respect to the source table by merging the materialized view and the source table.

Sharing materialized views in multiple tenant database systems

The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

Hidden database objects over external data

A system to identify optimal cloud resources for executing workloads. The system deduplicates historical client queries based on a workload selection configuration to determine a grouping of historical client queries. The system generates a workload based on at least a portion of the grouping of historical client queries. The system repeatedly executes a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run. The system, in response to determining that there is no performance difference, identifies one or more sets of decreased resources of the cloud environment. The system re-executes the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment.

Optimized identification of performance regression

Embodiments of the present disclosure may provide a dynamic query execution model. This query execution model may provide acceleration by scaling out parallel parts of a query (also referred to as a fragment) to additional computing resources, for example computing resources leased from a pool of computing resources. Execution of the parts of the query may be coordinated by a parent query coordinator, where the query originated, and a fragment query coordinator.

Scalable query processing

Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Interleaved execution infrastructure in database engines

A source table can be provided by a provider account in a data system. A secure view of the source table is provided to one or more consumer accounts, the secure view limiting access to a subset of data in the source table. A plan to execute a command using the secure view may be generated, the plan including a secure view boundary on a subset of operations defining the secure view. The plan may be modified to move a first operation that was outside the secure view boundary to within the secure view boundary to generate a second plan to optimize performance in view of limitations or restrictions placed by the secure view.

Performance optimizations for secure objects evaluations

Embodiments of the present disclosure may provide a data protection system that performs identification of errors from queries on a database. The data protection system can further identify corrupted data from additional errors, are difficult to detect, and occur between layers of data in the database system. The data protection system can perform corrections of the error data by rebuilding database data or removing the corrupted data.

Database metadata corruption mitigation

Methods, systems, and computer programs are presented for providing a cluster view method of a database to perform compaction and clustering of database objects, such as database materialized view. A cluster view system identifies a materialized view including data from one or more base tables, a portion of the data of the materialized view including stale data. The cluster view system performs an integrated task within a maintenance operation on a database, the integrated task including compacting the materialized view, the maintenance operation including clustering the materialized view, and stores the compacted and clustered materialized view in the database.

Clustering and compaction of materialized views on a database system

Joining data using a disjunctive operator is described. An example computer-implemented method can include generating, with a processing device, a query plan for a query, the query comprising a join operator expression for a disjunctive predicate, wherein the join operator expression includes a conjunctive predicate and a disjunctive operator. The method may further include generating a bloom filter for the disjunctive operator. Additionally, the method may include generating a result set as a result of evaluating the join operator expression using the disjunctive operator and bloom filter for the disjunctive predicate.

System and method for disjunctive joins

Growing community of inventors

San Mateo, CA, United States of America

Jiaqi Yan