Real-time anomaly prediction using extrapolated telemetry data

Abstract

Systems and methods are disclosed for real-time anomaly prediction using near real-time data. The invention addresses delays in telemetry data collection from infrastructure components, by collecting metrics and logging this data in real-time. Extracted logged data undergoes initial analysis to identify patterns and anomalies, followed by cleaning to remove noise and errors. Feature engineering enhances the data, creating or modifying features to improve machine learning model performance. The system calculates weighted means of previous data values and computes first and second-order differences to capture immediate changes and trends. These calculations adjust the extrapolated value to accurately reflect current conditions. The adjusted data is integrated into the dataset and validated. The validated data trains and tests a machine learning model, which is then finalized and deployed for real-time anomaly detection. This system ensures accurate and timely anomaly prediction, enabling automated incident response to maintain the reliability and performance of infrastructure components.