Splunk Inc.

Cisco Technology, Inc.

William Deaderick

Kristal Curtis

Tanner Gilligan

Poonam Yadav

Thomas Camp Vieth

William Stanton

Sichen Zhong

Joseph Ari Ross

Abraham Starosta

Chandrima Sarkar

Ralph Donald Thompson, Iii

Houwu Bai

Om Rajyaguru

Wei Jie Gao

Ramkumar Chandrasekharan

Ramprasad Siva Golla

Lila Fridley

Shailendra Suryawanshi

Aleksander Stojanovic

Alexander Stojanovic

Computerized methodologies are disclosed that are directed to detecting anomalies within a time-series data set. A first aspect of the anomaly detection process includes analyzing the regularity of the data points of the time-series data set and determining whether a data aggregation process is to be performed based on the regularity of the data points, which results in a time-series data set having data points occurring at regular intervals. A seasonality pattern may be determined for the time-series data set, where a silhouette score is computed to measure the quality of the fit of the seasonality pattern to the time-series data. The silhouette score may be compared to a threshold and based on the comparison, the seasonality pattern or a set of heuristics may be utilized in an anomaly detection process. When the seasonality pattern is utilized, the seasonality pattern may be utilized to generate thresholds indicating anomalous behavior.

Anomaly detection system and method for implementing a data regularity check and adaptive thresholding

Computerized methodologies are disclosed that are directed to detecting anomalies within a time-series data set. An aspect of the anomaly detection process includes determining one or more seasonality patterns that correspond to a specific time-series data set by evaluating a set of candidate seasonality patterns (e.g., hourly, daily, weekly, day-start off-sets, etc.). The evaluation of a candidate seasonality pattern may include dividing the time-series data set into a collection of subsequences based on the particular candidate seasonality pattern. Further, the collection of subsequences may be divided into clusters and a silhouette score may be computed to measure the clustering quality of the candidate seasonality pattern. In some instances, the candidate seasonality pattern having the highest silhouette score is selected and utilized in anomaly detection process. In other instances, a plurality of seasonality patterns may be combined forming a time policy, where the time policy is utilized in anomaly detection process.

Anomaly detection through clustering of time-series data subsequences and determination of adaptive thresholding

Systems and methods are disclosed that are directed to improving the prioritization, display, and viewing of system alerts through the use of machine learning techniques to group the alerts and further to prioritize the groupings. Additionally, a graphical user interface is generated that illustrates the prioritized listing of the plurality of groupings. Thus, a system administrator or other user receives an improved experience as the number of notifications provided to the system administrator are reduced due to the grouping of individual alerts into related groupings and further due to the prioritization of the groupings. Previously, or in current technology, system alerts may be automatically generated and provided immediately to a system administrator. In some instances, any advantage of detecting system errors or system monitoring provided by the alerts is negated by the vast number of alerts and provision of minimally important alerts in a manner that concealed more important alerts.

Machine-learning based prioritization of alert groupings

A computerized method is disclosed for grouping alerts through machine learning while implementing certain time constraints. The method includes receiving an alert to be assigned to any of a plurality of existing issues or to a newly created issue, the alert including a temporal field that includes a timestamp of an arrival time of the alert, wherein an issue is a grouping of one or more alerts, determining a subset of existing issues from the plurality of existing issues that each satisfy time constraints, wherein the time constraints correspond to (i) a time elapsed between a most recent alert of a first existing issue and a timestamp of the alert, or (ii) a maximum issue time length of the first existing issue, and deploying a trained machine learning model to assign the alert to either an existing issue of the subset of existing issues or a newly created issue.

Systems and methods for machine-learning based alert grouping including temporal constraints

Implementations of this disclosure provide an anomaly detection system and methods of performing anomaly detection on a time-series dataset. The anomaly detection may include utilization of a forecasting machine learning algorithm to obtain a prediction of points of the dataset and comparing the predicted value of a point in the dataset with the actual value to determine an error value associated with that point. Additionally, the anomaly detection may include determination of a sensitivity threshold that impacts whether points within the dataset associated with certain error values are flagged as anomalies. The forecasting machine learning algorithm may implement a seasonality component determination process that accounts for seasonality or patterns in the dataset. A search query statement may be automatically generated through importing the sensitivity threshold into a predetermined search query statement that implements that forecasting machine learning algorithm.

Hyperparameter tuning for anomaly detection service implementing machine learning forecasting

A computerized method is disclosed for grouping alerts through machine learning. The method including receiving an alert to be assigned to any of a plurality of existing issues or to a newly created issue, wherein an issue is a grouping of alerts, determining a temporal distance between the alert and each of the existing issues, determining either of (i) a numerical distance between the alert and each of the existing issues for a particular numerical field, or (ii) a categorical distance between the alert and each of the existing issues for a particular categorical field, determining an overall distance between the alert and each of the existing issues, and assigning the alert to either (i) an existing issue having a shortest overall distance to the alert that satisfies one or more time constraints, or (ii) the newly created issue.

Systems and methods for machine-learning based alert grouping

A computerized method is disclosed for grouping alerts and providing remediation recommendation. The method includes receiving the alert to be assigned to an existing open issue or a newly created issue, wherein an issue is a grouping of one or more alerts, assigning the alert to either a first existing open issue or the newly created issue by determining a weighted sum of the distance between the feature vectors of the alert and each existing open issue, determining a weighted sum of the distance between the feature vectors of the alert and each closed issue, and generating a user interface that illustrates an assignment of the alert and at least one of (i) a closed issue having a shortest distance to the alert or (ii) recommended remediation efforts associated with the closed issue having the shortest distance to the alert.

Systems and methods for machine-learning based alert grouping and providing remediation recommendations

A computerized method is disclosed for retraining machine learning models based on user feedback. The method includes receiving user feedback indicating a change is to be made to an assignment of one or more alerts, wherein the one or more alerts were assigned by a machine learning model implementing a distance metric, wherein an issue is a grouping of at least one alert, constructing a convex optimization procedure to minimize an adjustment of weights of the distance metric, retraining the machine learning model by adjusting the weights of the distance metric in accordance with the convex optimization procedure, and evaluating one or more subsequently received alert using the retrained machine learning model. Changes to be made to the assignment include any of merging of two issues, splitting of two issues based on time or an alert field, or reassignment of an alert from a first issue to a second issue.

Systems and methods for retraining machine-learning models to perform alert grouping

A computerized method is disclosed that includes operations of obtaining a data set, selecting candidate parameter pairs to be analyzed, wherein the candidate parameter pairs include a window length and a sensitivity multiplier, and wherein the window length is a number of data points, performing an anomaly detection process for each candidate parameter pair including importing each candidate parameter pair into a predetermined search query thereby generating a set of populated predetermined search queries, wherein the predetermined search query is configured to perform the anomaly detection process, executing each search query of the set of populated predetermined search queries on the data set to obtain a set of anomaly detection results, and scoring each anomaly detection result by applying a set of heuristics to the set of the anomaly detection results, and generating an auto-tuned search query by selecting a first candidate parameter pair based on a score of each of the set of anomaly detection results and importing the first candidate parameter pair into the predetermined search query.

System and method for automated determination of search query parameters for anomaly detection

A computerized method is disclosed for generating a prioritized listing of alerts based on scoring by a machine learning model and retraining the model based on user feedback. Operations of the method include receiving a plurality of alerts, generating a score for each of the plurality of alerts through evaluation of each of the plurality of alerts by a machine learning model, generating a prioritized listing of the plurality of alerts based on the generated scores, receiving user feedback on the prioritized listing, retraining the machine learning model based on the user feedback by generating a set of labeled alert pairs, wherein a labeled alert pair includes a first alert, a second alert, and an indication as to which of the first alert or the second alert is a higher priority in accordance with the user feedback, and evaluating subsequently received alerts with the retrained machine learning model.

System and method for machine-learning based alert prioritization

Systems and methods are described for training a machine learning (ML) model to group notable events reflecting operation of a computing system into episodes of related events reflecting an incident on the computing system, such as to enable root cause analysis of the incident. The ML model is trained using pairwise binary similarity labels (PBSLs) indicating that two events must or must not be grouped together. An interface is provided that facilitates rapid generating of PBSLs by relocating one or more events from a first episode to a second episode. The relocation input is translated into PBSLs that are then used to train the ML model.

Interface for incorporating user feedback into training of clustering model

Embodiments are directed to facilitating identifying seasonal frequencies. In particular, a set of candidate seasonal frequencies associated with a time series data set are determined based on ACF peaks identified in association with a representation of the time series data set. Thereafter, the filters are applied to analyze the candidate seasonal frequencies and update the candidate seasonal frequencies by removing any candidate seasonal frequencies that fail a filter. An example filter can include comparing ACF peaks with peaks associated with SDF peaks. Thereafter, a candidate seasonal frequency of the updated candidate seasonal frequencies can be identified as a seasonal frequency for the time series data set, and such a seasonal frequency can be provided (e.g., to a user or another process) for use in performing data analysis.

Growing community of inventors

Austin, TX, United States of America

William Deaderick