Snowflake Inc.

Marianne Shaw

Nileema Shingte

Saurin Shah

Vasile Paraschiv

Benoit Dageville

Thierry Cruanes

Subramanian Muralidhar

Istvan Cseri

Jiaqi Yan

Polita Paulus

Muthunagappan Muthuraman

Wai Sing Wong

Xinglian Liu

Selcuk Aya

Steven P Herbert

Marcia Feitel

James Malone

Nithin Mahesh

Dennis Huo

Ronald Lee Ortloff

The subject technology provides embodiments for supporting a unified table which may be a managed table or an unmanaged table. Managed tables are those where the subject technology manages the metastore/catalog for the table, whereas unmanaged tables are tables where an external catalog controls the table and the subject technology integrates with that catalog to work with the table, but does not assume control of the table.

Unified table data access in user-specified formats on internal storage and user-managed storage

A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

Export data from tables into partitioned folders on an external data lake

The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

Providing table data access in user-specified formats on user-managed storage

Growing community of inventors

Bothell, WA, United States of America

Marianne Shaw