Snowflake Inc.

Amazon Technologies, Inc.

Mihir Dharamshi

Cristian Diaconu

Chen Luo

Joshua Slocum

Wumengjian Zhu

Corbin McElhanney

Andrew McCormick

Alazel Acheson

Seth William Markle

Christopher Andrew Stephens

David J Carroll

Rajesh Sampath

Shreeranga Puttur Ramakrishna

Rameez Sajwani

Jonathan Herman

The subject technology receives, by an execution node, blob metadata from a key-value store, the blob metadata including information related to a set of blob files. The subject technology determines, by the execution node using the blob metadata, whether a copy of each of the set of blob files is stored in a local cache of the execution node. The subject technology transforms at least one blob file, retrieved from a blob store, to a second file in a column file format, the at least one blob file being in a first format that is different than the column file format, the transforming comprising at least converting a particular snapshot file from the at least one blob file to a particular set of rowsets and writing the set of rowsets into the second file in the column file format. The subject technology stores the second file in the local cache.

Columnar cache in hybrid transactional/analytical processing (HTAP) workloads

The subject technology receives a query, the query including a query range for processing the query and a set of requested columns. The subject technology based on the query range, determining a set of blob files and a set of delete vectors. The subject technology for each blob file, storing each row, including the set of request columns, into an array of rowsets. The subject technology for each rowset, generating a delete bitset to at least indicate whether each row has been deleted. The subject technology for each delta file, indicate a previous row of a visible row of the delta file as being deleted based on a delete pointer of the visible row. The subject technology providing a set of rowsets, including a corresponding selection column set, as a result of the query.

Scan-based merge for analytical query processing in HTAP systems using delete vectors

The subject technology receives a query, the query including a query range for processing the query. The subject technology sends a request to a key-value store for blob metadata and a set of recent writes for the query range. The subject technology receives the blob metadata, the blob metadata including information related to a set of blob files. The subject technology determines whether the set of blob files is stored in a local cache. The subject technology, in response to at least one blob file being missing from the set of blob files, sends a request to a blob store to retrieve the at least one blob file of the set of blob files. The subject technology transforms the retrieved at least one blob file to a column file format. The subject technology stores the transformed at least one blob file in the local cache.

A fleet node management system may include a metadata store, a plurality of fleet nodes, and one or more metadata mutation devices. The metadata store may be configured to store dynamic metadata. The plurality of fleet nodes may be configured to determine, based on a gossip protocol, whether to continue performance of a function that uses a local version of the metadata. The one or more metadata mutation devices may be configured to determine, based on a global state of the fleet nodes, whether to modify the dynamic metadata for the fleet nodes.

Fleet node management system

Growing community of inventors

Redmond, WA, United States of America

Mihir Dharamshi