Snowflake Inc.

John Reumann

Qiming Jiang

Orestis Kostakis

Boxin Jiang

Boyu Wang

Jiaxun Wu

Michel Adar

A system for improving task scheduling on a cloud data platform is provided. A task to be executed using resources of a computing cluster is received. A task execution plan is generated and information about data to be used for the ask is accessed. Resource requirements for executing the task are predicted by applying machine learning to the task execution plan and the information about the data. Assignment data is generated to execute the task on the resources by applying machine learning information about a current state of the resources and predicted resource requirements.

Task-execution planning using machine learning

Using an attributes model of a time series forecasting model, determine a set of features based on time series data, the set of features including periodic components. The time series data may be divided into a set of segments. For each segment of the set of segments, a weight may be assigned using an age of the segment, resulting in a set of weighted segments of time series data. Using a trend detection model of the time series forecasting model, trend data from the set of weighted segments of time series data may be determined. A time series forecast may be generated by combining the set of features and the trend data.

Enhanced time series forecasting

A system for improving task scheduling on a cloud data platform is provided. A task is received, from a user of a cloud data platform, for execution on a dataset of a cloud data platform using a plurality of resources. A task graph is generated, and metadata related to the dataset is accessed for use in execution of the task. A predicted resource profile is generated by applying a first machine learning scheme to the task graph and the metadata of the dataset. Assignment data is generated to execute processes of the task on the plurality of resources. The assignment data generated by applying a second machine learning scheme to current state data of a current computational state of the plurality of resources and the predicted resource profile generated by the first machine learning scheme.

Data-driven task-execution scheduling using machine learning

Embodiments of the present disclosure may provide a database optimization system that can generate computational values through a database compiler and assignment data for execution of a query by a plurality of nodes of a database system. The computational values and assignment data can be generated by one or more machine learning schemes. The machine learning schemes can be trained on previous computational values and previous assignment data.

Data-driven query-execution scheduling

A processing device may analyze a set of time series data using a time series forecasting model comprising an attributes model and a trend detection model. The attributes model may comprise a modified gradient boosting decision tree (GBDT) based algorithm. Analyzing the set of time series data comprises determining a set of features of the set of time series data, the set of features including periodic components as well as arbitrary components. A trend of the set of time series data may be determined using the trend detection model and the set of features and the trend may be combined to generate a time series forecast.

Growing community of inventors

Kirkland, WA, United States of America

John Reumann