Method and system for resource monitoring of large-scale, orchestrated, multi process job execution environments

Abstract

A system and method for monitoring the process resource consumption of massive parallel job executions is disclosed. The system uses byte code instrumentation to place sensors in methods that receive job execution requests. Those sensors detect start and end of job executions by the process they are deployed to and extract identification data from detected job execution requests that allow the monitor to identify the job request. This job identification data is used to tag resource utilization measures, which allows the monitor to assign measured resource consumptions to specific job executions. The job identification data that identifies the job execution that triggered the transaction is also used to tag transaction tracing data. The generated job specific measures and transaction traces may be used to identify resource intensive job executions and to identify the root cause of the resource consumption.