Snowflake Inc.

Oracle International Corporation

Snowflake Lnc.

Jiaqi Yan

Thierry Cruanes

Prasanna Rajaperumal

Allison Waingold Lee

Benoit Dageville

Igor Demura

William H Waddington

Varun Ganesh

Abdul Q Munir

Jeffrey Rosen

Libo Wang

Florian Andreas Funke

Shrainik Jain

Guangyan Hu

Stratis Viglas

Qiuye Jin

Yongsik Yoon

Nicola Dan Onose

Boyung Lee

Mohamed Zait

Ryan Michael Thomas Shelly

Bowei Chen

Bicheng Gao

Istvan Cseri

Marcin Zukowski

Dinesh Das

Sangyong Hwang

Srinivas Vemuri

Xuelai Cui

Jaeha Lee

Subramanian Muralidhar

Ganeshan Ramachandran Iyer

Li Yan

Yucan Liu

Shiyu Qu

Vlad Bunescu

Xinglian Liu

Selcuk Aya

Muthunagappan Muthuraman

Prashant Gaharwar

Zerui Wei

Nithin Mahesh

Niloy Mukherjee

Polita Paulus

Nileema Shingte

Sunil Parrokkaren Chakkappen

Yizhi Zhu

Yujie Li

Marianne Shaw

Nirav Vyas

Wai Sing Wong

Yan Huang

Vineet Garg

Leonid Baraznenok

James Malone

Marcia Feitel

Steven P Herbert

Adrian Peter Neumann

Wenhao Song

Saiyang Gou

Juan R Loaiza

Vineet Marwah

Tirthankar Lahiri

Damien Carru

Shasank Kisan Chavan

Jesse Kamp

David Schultz

Aurosish Mishra

Kartik Kulkarni

Yi Fang

Raja Suresh Krishna Balakrishnan

Jaeyun Noh

Brian Charles Forney

Xinzhu Cai

Nitish Jindal

Joshua Klahr

Jiaqi Zhang

Valeri Kim

Zhuo Peng

Prateek Gaur

Mengran Wang

Dmitry A Lychagin

Andrew Xue

Kevin Ye Jie Li

Karan Chadha

Alvin E Jou

Donghe Kang

Mohammad Saber Mirzaei

Weili Gu

Ronald Lee Ortloff

Dennis Huo

Louis Magarshack

Can Tuzla

Kavinder S Dhaliwal

Satyanarayana R Valluri

Harry Liu

A method for selecting micro-partitions for a clustering operation includes: storing table data in a plurality of micro-partitions of a storage device, wherein each of the plurality of micro-partitions comprises a portion of the table data, wherein subsets of the plurality of micro-partitions are associated with a respective one of a plurality of expression property (EP) files, and wherein each of the plurality of EP files comprises an EP data region that represents the portions of the table data of the subset of the plurality of micro-partitions associated with the EP file; determining sub-ranges of the table data based on the EP data regions of the plurality of EP files; selecting a subset of the plurality of EP files for a clustering operation based on the sub-ranges of the table data; and performing the clustering operation on the micro-partitions associated with the subset of the EP files.

Micro-partition clustering based on expression property metadata

The subject technology receives a query, the query referencing a unified representation for structured type data and semi-structured type data, the unified representation being provided in storage and in memory during query processing, the unified representation comprising a set of structured type fields that include a set of semi-structured typed fields that enables type safety and enforcement for the set of structured type fields, and flexibility for the set of semi-structured typed fields in a same column, the unified representation in storage including type information for the semi-structured type data as part of the semi-structured type data, the unified representation being utilized for structured type data and semi-structured type data. The subject technology processes the query using the unified representation stored in the memory, the unified representation providing performance parity between structured type data and semi-structured type data.

Unified structured and semi-structured data types in database systems

The subject technology receives a query, the query including a statement for performing the query. The subject technology performs a lookup operation on a stored plan cache based on the query. The subject technology performs, in response to a cache match of the query to a stored query plan in the stored plan cache based on the lookup operation, a validation process of the stored query plan. The subject technology determines whether the stored query plan is valid based on the validation process. The subject technology performs, in response to determining that the stored query plan is valid, a program building process for the stored query plan to generate a final query plan. The subject technology sends the final query plan to an execution node for execution.

Optimized validation of cached query plans

The subject technology receives a query. The subject technology performs a lookup operation on a shadow cache. The subject technology performs a validation process on the particular query plan for determining a cache hit on the shadow cache. The subject technology performs a program building process to update the particular query plan with additional information for executing the particular query plan. The subject technology performs a compilation process to compile an updated program of the updated particular query plan. The subject technology determines that the updated particular query plan is cacheable based at least in part on the compilation process. The subject technology registers the updated particular query plan as a new entry in the shadow cache. The subject technology sends the updated program to an execution node for executing the updated particular query plan.

Query plan cache shadow modes

Systems, methods, and devices for sharing materialized views in multiple tenant database systems. A method includes defining a materialized view over a source table that is associated with a first account of a multiple tenant database. The method includes defining cross-account access rights to the materialized view to a second account such that that second account can read the materialized view without copying the materialized view. The method includes modifying the source table for the materialized view. The method includes identifying whether the materialized view is stale with respect to the source table by merging the materialized view and the source table.

Sharing materialized views in multiple tenant database systems

The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

Hidden database objects over external data

A system to identify optimal cloud resources for executing workloads. The system deduplicates historical client queries based on a workload selection configuration to determine a grouping of historical client queries. The system generates a workload based on at least a portion of the grouping of historical client queries. The system repeatedly executes a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run. The system, in response to determining that there is no performance difference, identifies one or more sets of decreased resources of the cloud environment. The system re-executes the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment.

Optimized identification of performance regression

Embodiments of the present disclosure may provide a dynamic query execution model. This query execution model may provide acceleration by scaling out parallel parts of a query (also referred to as a fragment) to additional computing resources, for example computing resources leased from a pool of computing resources. Execution of the parts of the query may be coordinated by a parent query coordinator, where the query originated, and a fragment query coordinator.

Scalable query processing

Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Interleaved execution infrastructure in database engines

A source table can be provided by a provider account in a data system. A secure view of the source table is provided to one or more consumer accounts, the secure view limiting access to a subset of data in the source table. A plan to execute a command using the secure view may be generated, the plan including a secure view boundary on a subset of operations defining the secure view. The plan may be modified to move a first operation that was outside the secure view boundary to within the secure view boundary to generate a second plan to optimize performance in view of limitations or restrictions placed by the secure view.

Performance optimizations for secure objects evaluations

Embodiments of the present disclosure may provide a data protection system that performs identification of errors from queries on a database. The data protection system can further identify corrupted data from additional errors, are difficult to detect, and occur between layers of data in the database system. The data protection system can perform corrections of the error data by rebuilding database data or removing the corrupted data.

Database metadata corruption mitigation

Methods, systems, and computer programs are presented for providing a cluster view method of a database to perform compaction and clustering of database objects, such as database materialized view. A cluster view system identifies a materialized view including data from one or more base tables, a portion of the data of the materialized view including stale data. The cluster view system performs an integrated task within a maintenance operation on a database, the integrated task including compacting the materialized view, the maintenance operation including clustering the materialized view, and stores the compacted and clustered materialized view in the database.

Clustering and compaction of materialized views on a database system

Joining data using a disjunctive operator is described. An example computer-implemented method can include generating, with a processing device, a query plan for a query, the query comprising a join operator expression for a disjunctive predicate, wherein the join operator expression includes a conjunctive predicate and a disjunctive operator. The method may further include generating a bloom filter for the disjunctive operator. Additionally, the method may include generating a result set as a result of evaluating the join operator expression using the disjunctive operator and bloom filter for the disjunctive predicate.

System and method for disjunctive joins

The subject technology receives a query, the query including a statement for performing the query. The subject technology performs a first lookup operation on a multi-phase cache based on the query. The subject technology performs, in response to a first cache miss of the multi-phase cache, parsing of the statement from the query. The subject technology performs, based on the parsing, a compilation process on the query to generate a compiled query plan, the compilation process determining an optimization and a generalization for the query. The subject technology determines that the compiled query plan is cacheable. The subject technology registers, in response to the compiled query plan being cacheable, a dummy entry in the multi-phase cache.

Multi-phase query plan caching

Embodiments of the present disclosure may provide dynamic and fair assignment techniques for allocating resources on a demand basis. Assignment control may be separated into at least two components: a local component and a global component. Each component may have an active dialog with each other; the dialog may include two aspects: 1) a demand for computing resources, and 2) a total allowed number of computing resources. The global component may allocate resources from a pool of resources to different local components, and the local components in turn may assign their allocated resources to local competing requests. The allocation may also be throttled or limited at various levels.

Flexible computing

Growing community of inventors

San Mateo, CA, United States of America

Jiaqi Yan