System and method for evaluating and deploying unsupervised or semi-supervised machine learning models

Abstract

Methods of evaluating and deploying machine learning models for anomaly detection of a monitored system and related systems. Candidate machine learning algorithms are configured for anomaly detection of the monitored system. For each combination of candidate machine learning algorithm with type of anomalous activity, training and cross-validation sets are drawn from a benchmarking dataset. Using each of the training and cross-validation sets, a machine-learning model is trained and validated using the cross-validation set with average precision as a performance metric. A mean average precision value is then computed across these average precision performance metrics. A ranking value is computed for each candidate machine learning algorithm, and a machine learning algorithm is selected from the candidate machine learning algorithms based upon the computed ranking values. The selected machine learning model is deployed to a monitoring system that executes the deployed machine learning model to detect anomalies of the monitored system.