Microsoft Technology Licensing, LLC

Kexin Nie

Yang Yang

Zheng Li

Boyi Chen

Baolei Li

Shilpa Gupta

Yanbin Jiang

Technologies for monitoring performance of a machine learning model include receiving, by an unsupervised anomaly detection function, digital time series data for a feature metric; where the feature metric is computed for a feature that is extracted from an online system over a time interval; where the machine learning model is to produce model output that relates to one or more users' use of the online system; using the unsupervised anomaly detection function, detecting anomalies in the digital time series data; labeling a subset of the detected anomalies in response to a deviation of a time-series prediction model from a predicted baseline model exceeding a predicted deviation criterion; creating digital output that identifies the feature as associated with the labeled subset of the detected anomalies; causing, in response to the digital output, a modification of the machine learning model.

Machine learning model monitoring

Techniques for auto-tuning anomaly detection are provided. In one technique, training data is stored that comprises training instances, each of which comprises a severity-duration pair and a label that indicates whether the severity-duration pair represents an anomaly. A model is trained based on a first subset of the training data. A second subset of the training data is identified where each training instance includes a positive label that indicates that that training instance represents an anomaly. Based on the second subset of the training data, the model generates multiple scores, each of which corresponds to a different training instance. A minimum score is identified that ensures a particular recall rate of the model. In response to receiving a particular severity-duration pair, the model generates a particular score for the particular severity-duration pair. A notification of an anomaly is generated if the particular score is greater than the minimum score.

Auto-tune anomaly detection

Growing community of inventors

Sunnyvale, CA, United States of America

Kexin Nie