Snowflake Inc.

Garrett Frere

Pradeep Dorairaj

Boyu Wang

Carsten Stann

Mengjia Xia

The subject technology determines a set of shards of rows from a data set based on a number of rows and a number of execution nodes to execute a request for determining a correlation. For each shard from the set of shards, the subject technology sends a particular user defined table function (UDTF), including a particular shard of rows, to a different execution node to perform a set of operations for determining the correlation. The subject technology provides a set of output values of each particular UDTF corresponding to each shard from the set of shards in a second UDTF. The subject technology sends the second UDTF to a particular execution node to perform an aggregate operation using the set of output values of each particular UDTF. The subject technology receives a value of the correlation from the particular execution node based on the aggregate operation.

Distributed in-database vectorized operations using user defined table functions

The subject technology receives first data. The subject technology transforms first data to a wide format, the wide format comprising a second table. The subject technology splits a set of rows into a set of shards of rows. The subject technology, for each shard from the set of shards, sends a particular user defined table function (UDTF), including a particular shard of rows, to a different execution node to perform a set of operations for determining a rolling correlation over a window size. The subject technology provides a set of output values of each particular UDTF corresponding to each shard from the set of shards in a second UDTF. The subject technology sends the second UDTF to a particular execution node to perform an aggregate operation. The subject technology receives a value of the rolling correlation from the particular execution node based on the aggregate operation.

The subject technology receives a set of values for a data set corresponding to a request for determining a Pearson correlation. The subject technology determines a number of rows from the data set. The subject technology determines a number of execution nodes to execute the request. The subject technology determines a set of shards of rows from the data set. The subject technology, for each shard from the set of shards, sends a particular user defined table function (UDTF), including a particular shard of rows, to a different execution node to perform a set of operations. The subject technology provides a set of output values of each particular UDTF corresponding to each shard from the set of shards in a second UDTF. The subject technology sends the second UDTF to a particular execution node to perform an aggregate operation using the set of output values of each particular UDTF.

Growing community of inventors

Long Island City, NY, United States of America

Garrett Frere