Snowflake Inc.

Ryan Michael Thomas Shelly

Jiaqi Yan

Thierry Cruanes

Subramanian Muralidhar

Damien Carru

Nicola Dan Onose

Brian Smith

Jaeyun Noh

Varun Ganesh

Yizhi Zhu

Jaeha Lee

Zerui Wei

Leonid Baraznenok

Adrian Peter Neumann

Selcuk Aya

Yi Fang

Xinglian Liu

Harry Liu

Donghe Kang

Alvin E Jou

A method for selecting micro-partitions for a clustering operation includes: storing table data in a plurality of micro-partitions of a storage device, wherein each of the plurality of micro-partitions comprises a portion of the table data, wherein subsets of the plurality of micro-partitions are associated with a respective one of a plurality of expression property (EP) files, and wherein each of the plurality of EP files comprises an EP data region that represents the portions of the table data of the subset of the plurality of micro-partitions associated with the EP file; determining sub-ranges of the table data based on the EP data regions of the plurality of EP files; selecting a subset of the plurality of EP files for a clustering operation based on the sub-ranges of the table data; and performing the clustering operation on the micro-partitions associated with the subset of the EP files.

Micro-partition clustering based on expression property metadata

Techniques described herein can allow users to share cached results of an original query with other users while protecting sensitive information. The techniques described herein can check whether the other users have access to the underlying data queried before allowing those users to see the stored query results. That is, the system may perform privilege checks on the shared users before giving them access to the stored query results but without having to re-run the original query.

Privilege based access checks for query results

Embodiments of the present disclosure may provide a data protection system that performs identification of errors from queries on a database. The data protection system can further identify corrupted data from additional errors, are difficult to detect, and occur between layers of data in the database system. The data protection system can perform corrections of the error data by rebuilding database data or removing the corrupted data.

Database metadata corruption mitigation

Embodiments of the present disclosure describe systems, methods, and computer program products for improving query processing of a database. An example method can include: storing table data for a table in a plurality of micro-partitions, each micro-partition comprising a portion of the table data for the table; for each micro-partition of the plurality of micro-partitions, storing metadata for the micro-partition in at least one of a plurality of expression properties; and selecting, by a processing device, a subset of the plurality of expression properties to be grouped into a grouping expression property based at least partially on the metadata of the subset of the plurality of the expression properties. The grouping expression property may include cumulative metadata associated with the metadata of the subset of the plurality of expression properties.

Metadata clustering

The subject technology selects a most recently created file from a set of files stored in a source table. The subject technology iterates, in the source table, starting from the most recently created file up to an age threshold to select a first set of files for performing a first defragmentation process. The subject technology sets an indication corresponding to a particular file that is a last file, from the first set of files, that meets the age threshold. The subject technology performs the first defragmentation process on the selected first set of files. The subject technology determines that the first defragmentation process was successful.

File defragmentation service

Prevention and mitigation of corrupt database data

The subject technology receives a query plan corresponding to a query. The subject technology executes the query based at least in part on the query plan, the executing including: filtering a first set of files that are to be modified by a merge statement, performing a split operation to send information related to a second set of files to a scan set builder operation in a first portion of the query plan and scan back operation in a second portion of the query plan, performing the scan set builder operation to remove the second set of files from the first set of files, performing a table scan operation based on a third set of files, and performing a first union all operation to combine the first set of data with a second set of data as a first set of combined data.

Merge small file consolidation

Growing community of inventors

San Francisco, CA, United States of America

Ryan Michael Thomas Shelly