Accuracy regression detection for time series anomaly detection compute services

Abstract

Techniques for detecting regressions with respect to the accuracy of an anomaly detection compute service in detecting anomalies in users' time series data. The techniques include providing an instrumented time series instrumented with a set of one or more anomalies to the anomaly detection service. The anomaly detection service detects a set of one or more anomalies in the instrumented time series. The precision and recall of the detected anomalies with respect to the instrumented anomalies is computed. From the computed precision and recall, an anomaly detection accuracy is computed as an F-score or F-measure. It is then determined whether a regression in anomaly detection accuracy has occurred by comparing the computed accuracy score to a threshold. If a regression has occurred, an alert can be generated or a recent change to the anomaly detection service can be rolled back.