Composite model analysis of time series data having irregular trends for anomaly detection

Abstract

Hierarchical modelling and advanced feature engineering discover abnormalities in time series data with irregular trends. Data is collected in real time to ensure temporal integrity in the invention. Extraction filters and isolates useful data. Data cleansing removes noise and extraneous data after preliminary analysis identifies patterns and abnormalities. Feature engineering organizes cleansed data for machine learning algorithms. Primary storage stores this data for fast retrieval and extensive trend analysis. Holidays and weekends provide unique patterns in trend analysis. These trends are used to cluster data and create hierarchical predictive models, starting with a first-order model for general trends and increasing in order to refine residuals. Serializing these models improves storage and retrieval. Trend clusters are created from new data points, and algorithms detect pattern deviations. Statistical tests and machine learning classifiers identify anomalies and create alerts and remedial measures. The system monitors and analyzes incoming data to detect anomalies.