Informatica Corporation

Mohan Sankaran

Sankaran Suresh

Sanjeev K Gupta

Diaz Nesamoney

Frank Beier

Gautam H Mudunuri

Girish Pancha

Pinaki Mukhopadhyay

Frank Joseph DeRose

Jyotindra Pramathnath Gautam

Mon Wong

Volodymyr Butsky

Sen Ma

Kiran Bagepalli

Mohini Wettasinghe

Sridhar C Koritala

Andrew Comstock

Akin Dirik

Ivan Chong

Zhenyu Tang

A data profiling module receives user selection of spreadsheets, and the data from the selected spreadsheets is profiled. At least one matching column is identified among the spreadsheets selected. The data profiling module calculates a match metric for the at least one matching column, and unifies the spreadsheets into a single composite spreadsheet using the at least one identified matching column. A preview view of a composite spreadsheet is generated, visually indicating the at least one matching column, any non-matching columns between the spreadsheets, and the match metric for the matching columns. An action history module identifies spreadsheets for use in the procedure, and stores any action applied to the spreadsheets as a procedure template that can be applied to a plurality of other spreadsheets.

Composite data creation with refinement suggestions

A method and system providing a high speed and secure data link for moving large amounts of data across a network, such as the data used in an analytic application. Featured are simultaneous compression and encryption of the data, as well as means for recovery in the event the network connection is lost.

Method and system for providing transfer of analytic application data over a network

A method and system thereof for performing real time transformations of dynamically increasing databases is described. A session, identified as a real time session, is initialized. The real time session repeatedly executes a persistent (e.g., continually running) data transport pipeline of the analytic application. The data transport pipeline extracts data from a changing database, transforms the data, and writes the transformed data to storage (e.g., a data warehouse or data mart). The data transport pipeline is executed at the end of each time interval in a plurality of contiguous time intervals occurring during the real time session. The data transport pipeline remains running after it is executed, until the real time session is completed. Accordingly, new data are transformed in a timely manner, and processing resources are not consumed by having to repeatedly re-establish (re-initialize) the data transport pipeline.

Real time sessions in an analytic application

A computer software architecture to automatically optimize the throughput of the data extraction/transformation/loading (ETL) process in data warehousing applications. This architecture has a componentized aspect and a pipeline-based aspect. The componentized aspect refers to the fact that every transformation used in this architecture is built up with transformation components selected from an extensible set of transformation components. Besides simplifying source code maintenance and adjustment for the data warehouse users, these transformation components also provide these users the building blocks to effectively construct pertinent and functionally sophisticated transformations in a pipelined manner. Within a pipeline, each transformation component automatically stages or streams its data to optimize ETL throughput. Furthermore, each transformation either pushes data to another transformation component, pulls data from another transformation component, or performs a push/pull operation on the data. Thereby, the pipelining; staging/streaming; and pushing/pulling features of the transformation components effectively optimizes the throughput of the ETL process.

Method and architecture for automated optimization of ETL throughput in data warehousing applications

A method for updating a target table of a data mart in response to changes made by a transaction to data stored in a source table of an operational database. Data that was changed in the source table by the transaction is stored in a dynamic image table of a change capture database. Data that was not changed in the source table by the transaction, but which is nevertheless required to be mapped to the target table, is stored in a static image table of the change capture database. The change capture database also contains relevant information regarding the transaction. Once the dynamic and static image tables are properly staged, the changes are propagated from the change capture database to the target tables of the data marts. In other words, data is extracted from the change capture database and subsequently transformed and loaded, thereby minimizing the impact to the operational database. Thereupon, the tables of the change capture database are truncated to discard data which is now no longer needed.

Apparatus and method for capturing and propagating changes from an

A method of performing incremental aggregation of dynamically increasing database data sets. An embodiment of the present invention operates within a data mart or data warehouse to aggregate data stored within an operational database corresponding to newly received data to provide current information. Initially, a computer server creates an intermediate file which is initialized by the server with an aggregate data set. The aggregate data set consists of data values and count values that each correspond to specific group identifiers. The computer determines if any group identifiers within a new set of inputs data are identical to any group identifiers stored within the intermediate file. If an inputted group identifier matches a stored group identifier, the inputted data value is aggregated with the stored data value and the count value corresponding to the specific stored group identifier is incremented by one. If an inputted group identifier does not match any of the stored group identifiers, the inputted group identifier and corresponding data value are stored within the intermediate file and a count value of one is appended to that specific group identifier. Once all the group identifiers within the new set of input data have been determined, the computer stores all the changes that were made to the intermediate file into the aggregate data set.

Method for incremental aggregation of dynamically increasing database

Growing community of inventors

Union City, CA, United States of America

Mohan Sankaran