Amazon Technologies, Inc.

James P Bartlett

Peter Sirota

Richard Jeffrey Cole

Adam David Gray

Vaibhav Aggarwal

Kathryn Marie Shih

Carl Louis Christofferson

Eider Brantly Moore

Richard Rex McKnight

Adam D Gray

Richard J Cole

Lorenzo Minore

Techniques are described for providing clients with access to functionality for creating, configuring and executing defined workflows that manipulate source data in defined manners, such as under the control of a configurable workflow service that is available to multiple remote clients over one or more public networks. A defined workflow for a client may, for example, include multiple interconnected workflow components that are specified by the client and that each are configured to perform one or more types of data manipulation operations on a specified type of input data. The configurable workflow service may further execute the defined workflow at one or more times and in one or more manners, such as in some situations by provisioning multiple computing nodes provided by the configurable workflow service to each implement at least one of the workflow components for the defined workflow.

Providing configurable workflow capabilities

Methods and systems for using a scheduler in a data pipeline are disclosed. A plurality of objects in a first layer are created, each representing a respective regularly scheduled task. A plurality of objects in a second layer are created, each representing a respective scheduled instance of a regularly scheduled task. It is determined whether each object in the second layer is ready to execute. For at least one object in the second layer, it is determined if the object has received notifications from any objects on which it depends. For each object that is ready to execute, the regularly scheduled task associated with the object is performed. For each object that is not ready to execute, the object is put to sleep.

Scheduler for data pipeline

Methods and systems for a connector interface in a data pipeline are disclosed. A pipeline comprising two data source nodes and an activity node is configured. Each data source node represents data from a different data source, and the activity node represents a workflow activity that uses the data as input. Two connectors which implement the same connector interface are triggered. In response, data is acquired at each connector from the corresponding data source through the connector interface. The data is sent from the connectors to the activity node through the connector interface. The workflow activity is performed using the acquired data.

Connector interface for data pipeline

Growing community of inventors

Seattle, WA, United States of America

James P Bartlett