System and method for data drift detection while performing anomaly detection

Abstract

Methods and systems for detecting data drift while performing anomaly detection in a distributed environment are disclosed. To perform anomaly detection, a system may include an anomaly detector and one or more data collectors. The anomaly detector may detect anomalies in data obtained from one or more of the data collectors using a continuous inference model. To detect data drifts in data from the one or more data collectors, the anomaly detector may also detect anomalies in data obtained from one or more data detectors using a quantized inference model. The output of the continuous inference model may be compared to the output of the quantized inference model to determine whether the continuous inference model has adapted to data drift over time through re-training. Following anomaly detection and/or data drift detection, the data may be discarded to remove the data from the anomaly detector.