Evolution of topics in a messaging system

Abstract

Systems and methods for determining how topics evolve in a messaging system extract at least one N-gram from data content (e.g., caption of messages) in the messaging system and detect anomalous behavior in N-gram frequencies over time. The anomalous behavior is used to select candidate N-grams for a determination of whether a topic of a candidate N-gram is evolving or fading. The candidate N-grams are clustered into cluster groups that are used to train at least one time series forecasting model to predict N-gram frequencies in a future time window. A time series of the N-gram frequency is divided into old and recent partitions and pattern recognition is applied to the predicted N-gram frequencies to identify an evolving or fading topic when the difference between a frequency of each anomaly and an average rolling median for each partition is greater for the most recent partition.