Streamsets, Inc.

Madhukar Devaraju

Arvind Prabhakar

Alejandro Humberto Abdelnur

Systems and methods are directed to remote validation and preview. An example system receives an indication of a portion of the data pipeline to be processed, generates a data pipeline configuration file describing operations in the portion of the data pipeline, causes a software framework to perform operations corresponding to the portion of the data pipeline, receives results of the operations corresponding to the portion of the data pipeline, and causes presentation of the results on a graphical user interface of a computing device.

Remote validation and preview

In various example embodiments, a system, computer readable medium and method for schema update engine dynamically updating a target data storage system. Incoming data records are received. A front-end schema of the incoming data records is identified. The front-end schema and the current target schema are compared. Based on identifying a difference between the front-end schema and the current target schema, the current target schema is updated in order to be identical to the front-end schema. The current target data file is closed and the incoming data records are stored in a new target data file according to the updated target schema.

Growing community of inventors

San Francisco, CA, United States of America

Madhukar Devaraju