Generating test data for application performance

Abstract

In an approach to improve the extraction of test datasets for testing and resource optimization, embodiments execute a test run on a full dataset of a job, and identify existing bottlenecks in the job through a run-time monitoring interface. Additionally, embodiments execute a run-time metrics analysis, a source code analysis, and a source data impact analysis of a distributed data processing engine executing a distributed data processing job, and generate, by an analysis and impact scoring engine, an impact scoring table of job transformations based on the source code analysis. Furthermore, embodiments generate data extraction rules based on the impact scoring table, and extract a test dataset based on the data extraction rules. Moreover, embodiments evaluate the data extraction rules against user defined thresholds, and prepare a representative test dataset, and output, through a user interface on a computing device, the representative test dataset to a user.