System and method for artificial intelligence base prediction of delays in pipeline processing

Abstract

A method and system are provided for training a machine learning (ML) system for predicting delays in processing pipelines. In one embodiment, the method includes receiving labelled historical data pertaining to a pipeline, the labelled data including trigger objects initiating the pipeline and one or more processing times corresponding to one or more stages of the pipeline. The method includes identifying features associated with the trigger objects, formatting the labelled data and, randomly splitting the formatted labelled data into a full training dataset and a testing dataset. Additionally, the method includes distributing the full training dataset into several partial datasets and, in an ensemble ML system, training each of several ML subsystems using a respective partial dataset to provide a respective individual inference model predicting respective processing times, and deriving and storing an ML model for prediction of delays by aggregating the individual inference models.