Managing data drift and outliers for machine learning models trained for image classification

Abstract

A system and a method for updating a Machine Learning (ML) model are described The method involves capturing reconstruction errors associated with reconstruction of images by a pre-trained autoencoder. Data points representing the reconstruction errors are clustered using affinity propagation. A preference value used by the affinity propagation for determining similarity between the data points is dynamically set through linear regression. Outliers and data drifts are determined from clusters of the data points. Classification output of the ML model is associated with the outliers and the data drift, for refinement of the ML model over a device hosting a training environment.