Anomaly detection based preprocessing for better classification tasks with noisy and imbalanced datasets

Abstract

A computer trains, based on many timeseries, many anomaly detectors. Each anomaly detector is configured with a respective distinct contamination factor. Each timeseries is a temporal sequence of datapoints that characterize a device. Each datapoint in the many timeseries has a respective label that indicates whether the device failed when the datapoint occurred. Each anomaly detector detects: a set of anomalous datapoints, the size of which is proportional to the contamination factor of the anomaly detector, a healthy count of anomalous datapoints in timeseries of devices not failed, and an unhealthy count of anomalous datapoints in timeseries of failed devices. For a particular anomaly detector, the computer detects that the magnitude of the difference between the respective healthy count and the respective unhealthy count is less than a predefined threshold. Based on the contamination factor of the particular anomaly detector, anomalous datapoints are oversampled. Based on the oversampled anomalous datapoints, a classifier is trained.