International Business Machines Corporation

Anyi Li

Scott J McCallen

Kaveh Noorbakhsh

Douglas S Meil

Timothy M Fox

Patrick Sullivan

Matthew J Wollerman, Ii

Christopher L Crabtree

A method, a processing device and a computer program product are provided. Based on parameters pertaining to a data source, a first processing device may determine whether a data source is a known data source. If the data source is determined to be the known data source, then the first processing device sends respective configuration information to multiple processing devices. The respective configuration information being based on configuration information from a previous integration and configuring the multiple processing devices as an operational data pipeline to extract data from the known data source, transform the extracted data to a first data model employed by a data system, and load the transformed data into the data system.

Automating configuration of operational data pipelines for extraction, transformation and load

A system processes streaming data and includes at least one processor. The system may write streaming data received from a data source as messages in queues at a queuing cluster. The queuing cluster includes a coordinator node to direct the messages to non-coordinator nodes of the queuing cluster. The system may retrieve the data from the queues based on subscription of topics and store the retrieved data in a consumable repository.

Scalable event stream data processing using a messaging system

Growing community of inventors

Rocky River, OH, United States of America

Anyi Li