Population anomaly detection through deep gaussianization

Abstract

Anomalies in a data set may be difficult to detect when individual items are not gross outliers from a population average. Disclosed is an anomaly detector that includes neural networks such as an auto-encoder and a discriminator. The auto-encoder and the discriminator may be trained on a training set that does not include anomalies. During training, an auto-encoder generates an internal representation from the training set, and reconstructs the training set from the internal representation. The training continues until data loss in the reconstructed training set is below a configurable threshold. The discriminator may be trained until the internal representation is constrained to a multivariable unit normal. Once trained, the auto-encoder and discriminator identify anomalies in the evaluation set. The identified anomalies in an evaluation set may be linked to transaction, security breach or population trends, but broadly, disclosed techniques can be used to identify anomalies in any suitable population.