Automated and customized post-production release review of a model

Abstract

A non-transitory computer readable medium includes computer executable instructions that, when executed, cause at least one processor to train a model to perform at least one of a prediction operation, a diagnostic operation, or a classification operation based on a training dataset, deploy the model in a production computer system to perform the at least one operation on field data, monitor signal data associated with the model, the signal data including specific or derived signal data representing characteristics of an ecosystem in which the model is deployed and new observations in incoming field data, monitor accuracy of the model by applying a statistical tool to a plurality of data points of the signal data, apply a secondary machine learning predictive engine to the plurality of data points of the signal data to predict future data points of the signal data, determine whether the signal data represents an unstable process by identifying future outlier data points from among the plurality of future data points of the signal data, select a rule corresponding to the future outlier data points, the rule to suggest at least one of a cause of the unstable process or an effect of the unstable process on the signal data, and generate an indication that a corrective action should be taken on the model based on a result of the determination, the indication to identify the rule and at least one of the cause of the unstable process or the effect of the unstable process on the signal data.