Framework for measuring telemetry data variability for confidence evaluation of a machine learning estimator

Abstract

A deployment manager includes storage for storing a prediction model based on telemetry data from the deployments and a prediction manager. The prediction manager generates, using the prediction model and second telemetry data obtained from a deployment of the deployments: a prediction, and a prediction error estimate; in response to a determination that the prediction indicates a negative impact on the deployment: generates a confidence estimation for the prediction based on a variability of the second telemetry data from the telemetry data; in response to a second determination that the confidence estimation indicates that the prediction error estimate is inaccurate: remediates the prediction based on the variability to obtain an updated prediction; and performs an action set, based on the updated prediction, to reduce an impact of the negative impact on the deployment.