Efficient transformer for content-aware anomaly detection in event sequences

Abstract

A method for implementing a self-attentive encoder-decoder transformer framework for anomaly detection in event sequences is presented. The method includes feeding event content information into a content-awareness layer to generate event representations, inputting, into an encoder, event sequences of two hierarchies to capture long-term and short-term patterns and to generate feature maps, adding, in the decoder, a special sequence token at a beginning of an input sequence under detection, during a training stage, applying a one-class objective to bound the decoded special sequence token with a reconstruction loss for sequence forecasting using the generated feature maps from the encoder, and during a testing stage, labeling any event representation whose decoded special sequence token lies outside a hypersphere as an anomaly.