International Business Machines Corporation

Other

Raghavendra Rao Dhayapule

Ritesh Kumar Gupta

Sumit Kumar

Rajesh Phillips

Jean-Claude Mamou

Yeh-Heng Sheng

Natesh H Mariyappa

Ajay Kumar Sood

Srinivas R Mangi

Patrick A Dass1

Patrick A Dass

Shrinidhi Atmakur

A system may generate a web conference recording of a live web conference among a plurality of participant computing systems. The system may receive, at the end of the live web conference, one or more local recordings from one or more participant computing systems of the plurality of participant computing systems. A system may merge content from the one or more local recordings into the web conference recording, the content from the one or more local recordings corresponding to one or more connectivity issues of a corresponding participant computing system during the live web conference.

Web conference recording content merging

A distributed network of participating Extract Transform Load (ETL) servers is received. Data source mappings are generated for the distributed network, where the data source mappings indicate which participating ETL servers in the distributed network have access to which tables in data sources. Network metrics are obtained that indicate, for each pair of participating ETL servers, an average data transmission speed and a unit cost. Data source metrics are obtained for the tables in the data sources. A link mappings table is generated that lists mappings of each link to a network in between participating ETL servers. A plurality of distributed execution plans are generated using the network metrics, the data source metrics, and the link mappings table. An execution plan is selected from the plurality of execution plans according to an optimization criteria. The selected execution plan is executed.

Distributed balanced optimization for an Extract, Transform, and Load (ETL) job

In deploying a system in a computing environment, before deployment, a deployment engine performs a pre-deployment validation of the system using pre-determined system requirements. When the pre-deployment validation of the system fails, the deployment of the system is terminated. When the pre-deployment validation of the system succeeds, the deployment engine performs the following for each artifact of the system. Before deployment of a given artifact of the system, a pre-deployment validation of the given artifact is performed using pre-determined artifact properties. When the pre-deployment validation of the given artifact fails, the deployment of the system is terminated. When the pre-deployment validation of the given artifact succeeds, the given artifact is deployed. After the deployment of the given artifact, a post-deployment validation of the given artifact is performed using deployment data for the given artifact.

Distributed product deployment validation

A method for testing an extract, transform, load job (ETL job) is provided. The method may include receiving an uploaded ETL job. The method may also include executing a functionality test of the uploaded ETL job using a test bucket. The method may further include determining the executed functionality test produces a failure result. The method may also include creating at least one test sub-job corresponding to at least one stage within on the uploaded ETL job. The method may further include generating a plurality of test data for each of the at least one created test sub-job. The method may also include executing the at least one created test sub-job using the plurality of generated test data. The method may further include creating a report based on a plurality of results associated with the at least one executed test sub-job.

Automating extract, transform, and load job testing

Compiling test cases. The method may include uploading at least one extract, transform, load job (ETL job) and a plurality of test cases. The method may also include identifying a plurality of design features associated with the at least one uploaded ETL job. The method may further include identifying at least one feature change within the plurality of identified design features. The method may also include determining a plurality of related test cases within the plurality of uploaded test cases corresponding to the at least one identified feature change. The method may further include creating a test bucket based on a grouping of the determined plurality of related test cases. The method may also include adding at least one of a plurality of corresponding ETL jobs and a plurality of corresponding ETL job stages associated with the at least one identified feature change to the created test bucket.

Compiling extract, transform, and load job test data cases

Provided are techniques for distributed balanced optimization for an Extract, Transform, and Load (ETL) job across distributed systems of participating ETL servers using a data flow graph with links and stages for an ETL job to be executed by participating ETL servers is received. A distributed job execution plan is generated that breaks the data flow graph into job segments. The job segments each include a subset of the links and stages and map to one participating ETL server from the distributed systems to meet an optimization criteria across the distributed systems. The job segments are distributed to the participating ETL servers based on the mappings for parallel execution. Also, the distributed job execution plan utilizes statistics to reduce data movement and redundancies and to balance workloads across the distributed systems.

Distributed balanced optimization for an extract, transform, and load (ETL) job

Handling extract-transform-load (ETL) job mismatches as 'exceptions.' Exception handling may include the following steps: (i) determining a mismatch while running an extract-transform-load job with the mismatch being a mismatch of at least one of the following types: design time information mismatch, and/or operational metadata mismatch; and (ii) responsive to determining the mismatch, handling the mismatch as an exception.

Growing community of inventors

Bengaluru, India

Raghavendra Rao Dhayapule