Oracle International Corporation

Vikas Varma

Sachin Sadashiv Thatte

David Allan

Arun Patnaik

Frank Joseph Klein

Sebu Thomas Koleth

Sathish Paul Leo

Techniques describes herein updating pipeline flows in within data systems to maintain data integrity and consistency without manual curation. In certain embodiments, data integration system may detect and/or receive indications of a schema change within a source system of the data integration system. One or more data objects affected by the schema change may be identified, and a set of pipeline rules may be invoked for each of the affected schema changes. The pipeline rules may define a single transformation or a multi-step transformation process by which the data in the source system is provided to one or more target systems. After applying the pipeline rules to the updated source schema, the data received from the source system may be processed using the updated pipeline rules, transformed, and transmitted to the target system(s) to maintain the data integrity of the system.

Self-correcting pipeline flows for schema drift

In various embodiments, a data integration system is disclosed which enables users to debug distributed data integration scenarios which are platform and technology independent. A debugger client can connect to a plurality of local and/or remote hosts executing portions of a distributed data integration scenario. The debugger client can additionally enable line-by-line debugging of the portions of the distributed data integration scenario using a plurality of language-specific interfaces. The language-specific interfaces can further enable the user to dynamically update and debug changes to the code during debugging, reducing the time and resources required by multiple recompilations of the code.

Debugging framework for distributed ETL process with multi-language support

Growing community of inventors

Sunnyvale, CA, United States of America

Vikas Varma