Vmware, Inc.

Darren Brown

Junyuan Lin

Nicholas Kushmerick

Paul Pedersen

Xing Wang

Peng Gao

Keshav Mathur

Leah Nutman

Jinyi Lu

Matt Roy McLaughlin

Jeremy OlmstedThompson

Anil Sharma

Wei Ey Li

Ashok Kumar

Mayank Agarwal

Ron Oded Gery

Apolak Borthakur

Jon Herlocker

Sergio Nakai

Dhaval Gada

Gopal Harikumar

Pedro Algarvio

Caleb Beard

Shafi Khan

Computational methods and systems to detect anomalous behaving resources and objects of a distributed computing system are described. Multiple streams of metric data representing usage of various resources of the distributed computing system are sent to a management system of the distributed computing system. The management system updates a performance model based on newly received metric values of the streams of metric data. The updated performance model is used to detect changes in one or more of the streams of metric data. The changes may be an indication of anomalous behavior at resources and objects associated with the streams of metric data. An anomaly listener is notified of anomalous behavior by the resource or object when a change in one or more of the streams of metric data is detected.

Streaming anomaly detection

Computer-implemented processes and systems described herein are directed to reducing volumes of data sent from edge devices to a data center. Each edge device runs an agent that collects event information generated by event sources of the edge device in a runtime interval. Each agent reduces the event information to relevant event information at the edge device in accordance with instructions received from a controller server of the data center. The relevant event information contains less information than the event information. Each agent forwards the relevant event information over the internet to external services executed at the data center, where the relevant event information is stored in a data storage device of the data center.

Processes and systems for decentralization of data produced at the edge of a distributed computing system

Examples herein include systems and methods for providing capacity forecasting for high-usage periods of a computing infrastructure. An example method can include segmenting a first portion of a data stream and generating a first core set for a forecasting model that predicts future usage of computing resources. The example method can further include segmenting a second portion of the data stream, generating a second core set, and using both core sets to forecast usage. The first core set can then be phased out after a predetermined time period has elapsed such that forecasting is based only on the second core set. The example method can further include defining at least two clusters of data and performing predictive analysis on that specific cluster. Cluster-specific results can be displayed on a GUI, which can also provide a user with options for increase or decrease computing resources based on the predictions.

Capacity forecasting for high-usage periods

Various examples are disclosed for transitioning usage forecasting in a computing environment. Usage of computing resources of a computing environment are forecasted using a first forecasting data model and usage measurements obtained from the computing resources. A use of the first forecasting data model in forecasting the usage is transitioned to a second forecasting data model without incurring downtime in the computing environment. After the transition, the usage of the computing resources of the computing environment is forecasted using the second forecasting data model and the usage measurements obtained from the computing resources. The second forecasting data model exponentially decays the usage measurements based on a respective time period at which the usage measurements were obtained.

Exponential decay real-time capacity planning

Computer-implemented methods and systems described herein are directed to reducing volumes of log messages sent from edge systems to a data center. The computer-implemented methods performed at each edge system includes collecting a stream of log messages generated by one or more event sources of the edge system. Representative log messages of the stream of log messages are determined. The edge systems may discard non-representative log messages from data storage devices at the edge system. The representative log messages are sent from each of the edge systems to the data center where the representative log messages are received and stored in data storage devices of the data center, thereby reducing the volumes of log messages sent from the edge systems to the data center.

Methods and systems for reducing volumes of log messages sent to a data center

Computational methods and systems for detecting and troubleshooting anomalous behavior in distributed applications executing in a distributed computing system are described herein. Methods and systems discover nodes comprising the application. Anomaly detection monitors the metrics associated with the nodes for anomalous behavior in order to identify an approximate point in time when anomalous behavior begins to adversely impact performance of the application. Anomaly detection also monitors logs messages associated with the nodes to detect anomalous behavior recorded in the log messages. When anomalous behavior is detected in either the metrics and/or the log messages an alert identifying the anomalous behavior is generated. Troubleshooting guides an administrator and/or application owner to investigate the root cause of the anomalous behavior. Appropriate remedial measures may be determined based on the root cause and automatically or manually executed to correct the problem.

Methods and systems for troubleshooting applications using streaming anomaly detection

The current document is directed to methods and systems that process, classify, efficiently store, and display large volumes of event messages generated in modern computing systems. In a disclosed implementation, received event messages are assigned to event-message clusters based on non-parameter tokens identified within the event messages. A parsing function is generated for each cluster that is used to extract data from incoming event messages and to prepare event records from event messages that more efficiently and accessible store event information. The parsing functions also provide an alternative basis for assignment of event messages to clusters. Event types associated with the clusters are used for gathering information from various information sources with which to automatically annotate event messages displayed to system administrators, maintenance personnel, and other users of event messages.

Method and system for clustering event messages and manage event-message clusters

Automated methods and systems described herein are directed to identifying potential root causes of a problem in a data center. Methods and systems receipt an alert or other notification of a problem occurring in a data center and a time when the problem was noticed. A search window is created based on the time and a stream of log messages generated in the search window is converted into a time dependent metric. An anomaly detection technique is applied to the metric to determine a start time of a problem. Logging events and key phrases in the log messages are identified in the search window and presented as potential root causes of the problem. The potential root cause may then be used by system administrators and/or tenants to diagnose the problem and execute remedial measures to correct the problem.

Methods and systems for determining potential root causes of problems in a data center using log streams

The current document is directed to systems, and methods incorporated within the systems, that carry out probability-distribution-based analysis of log-file entries. A monitoring subsystem within a distributed computer system uses probability-distribution-based analysis of log-file entries to detect changes in the state of the distributed computer system. A log-file-analysis subsystem within a distributed computer system uses probability-distribution-based analysis of log-file entries to identify subsets of log-file entries that predict anomalies and impending problems in the distributed computer system. In many implementations, a numerical comparison of probability distributions of log-file-entry types is used to detect state changes in the distributed computer system.

Probability-distribution-based log-file analysis

Various examples are disclosed for forecasting resource usage and computing capacity utilizing an exponential decay. In some examples, a computing environment can obtain usage measurements from a data stream over a time interval, where the usage measurements describe utilization of computing resource. The computing environment can generate a weight function for individual ones of the usage measurements, where the weight function exponentially decays the usage measurements based on a respective time period at which the usage measurements were obtained. The computing environment can forecast a future capacity of the computing resources based on the usage measurements and the weight function assigned to the individual ones of the usage measurements. The computing environment can further upgrade a forecast engine to use the exponential decay without resetting the forecast engine or its memory.

Computational methods and systems described herein are directed to compounding alerts generated in a distributed computing system. A user or system administrator may define a set of multistage process rules that can be used by a log management server application to examine log messages generated by event sources of a multistage process for alerts. A log-message database is searched to identify a log-message file used to record log messages generated by the event sources. A single compound alert indicating that the multistage process rules are satisfied is generated, when log messages of the log-message file that satisfy the rules of the multistage process rules have been identified. Methods may also execute remedial action to correct the multistate process when log messages of the log-message file fail to satisfy at least one rule of the multistage process rules.

Methods and systems to compound alerts in a distributed computing system

Computational methods and systems described herein manage alerts generated by event sources that run in a distributed computing system. Methods and system provide a graphical user interface that enables a user to define a dominant alert and select subsumed alerts generated by the event sources. Methods and systems may also compute a relative fraction that represents a number of times each alert is triggered with respect to a number of times another alert is triggered for each pair of alerts. The relative fractions may be displayed in the graphical user interface to allow a user to select dominant and subsumed alerts based on the relative fractions. Methods and systems identify log messages that correspond to user-identified subsumed alerts, suppress subsumed alerts and generate the dominant alert. Methods and systems may also execute remedial action to correct the problem represented by the dominant alert.

Methods and systems to manage alerts in a distributed computing system

Computational methods and systems that proactively manage usage of computational resources of a distributed computing system are described. A sequence of metric data representing usage of a resource is detrended to obtain a sequence of non-trendy metric data. Stochastic process models, a pulse wave model and a seasonal model of the sequence of non-trendy metric data are computed. When a forecast request is received, a sequence of forecasted metric data is computed over a forecast interval based on the estimated trend and one of the pulse wave or seasonal model that matches the periodicity of the sequence of non-trendy metric data. Alternatively, the sequence of forecasted metric data is computed based on the estimated trend and the stochastic process model with a smallest accumulated residual error. Usage of the resource by virtual objects of the distributed computing system may be adjusted based on the sequence of forecasted metric data.

Methods and systems to proactively manage usage of computational resources of a distributed computing system

The current document is directed to systems, and methods incorporated within the systems, that execute queries against log-file entries. A monitoring subsystem within a distributed computer system uses query results during analysis of log-file entries in order to detect changes in the state of the distributed computer system, identify problems or potential problems, and predict and forecast system characteristics. Because of the large numbers of log-file-entry containers that may need to be opened and processed in order to execute a single query, and because opening and reading through the entries in a log-file-entry container is a computationally expensive and time-consuming operation, the currently disclosed systems employ event-type metadata associated with log-file-entry containers to avoid opening and reading through the log-file entries of log-file-entry containers that do not contain log-file entries with event types relevant to the query.

Growing community of inventors

Seattle, WA, United States of America

Darren Brown