Vmware, Inc.

Marklogic Corporation

Paul Pedersen

Darren Brown

Xing Wang

Junyuan Lin

Keshav Mathur

Leah Nutman

Peng Gao

Jinyi Lu

Christopher Lindblad

Wei Ey Li

Nicholas Kushmerick

Yu P Sun

Apolak Borthakur

Sandy Lau

Sergio Nakai

Gopal Harikumar

Shafi Khan

Parikshit Santhana Gopalan Gopalan

Computational methods and systems to detect anomalous behaving resources and objects of a distributed computing system are described. Multiple streams of metric data representing usage of various resources of the distributed computing system are sent to a management system of the distributed computing system. The management system updates a performance model based on newly received metric values of the streams of metric data. The updated performance model is used to detect changes in one or more of the streams of metric data. The changes may be an indication of anomalous behavior at resources and objects associated with the streams of metric data. An anomaly listener is notified of anomalous behavior by the resource or object when a change in one or more of the streams of metric data is detected.

Streaming anomaly detection

Examples herein include systems and methods for providing capacity forecasting for high-usage periods of a computing infrastructure. An example method can include segmenting a first portion of a data stream and generating a first core set for a forecasting model that predicts future usage of computing resources. The example method can further include segmenting a second portion of the data stream, generating a second core set, and using both core sets to forecast usage. The first core set can then be phased out after a predetermined time period has elapsed such that forecasting is based only on the second core set. The example method can further include defining at least two clusters of data and performing predictive analysis on that specific cluster. Cluster-specific results can be displayed on a GUI, which can also provide a user with options for increase or decrease computing resources based on the predictions.

Capacity forecasting for high-usage periods

Various examples are disclosed for transitioning usage forecasting in a computing environment. Usage of computing resources of a computing environment are forecasted using a first forecasting data model and usage measurements obtained from the computing resources. A use of the first forecasting data model in forecasting the usage is transitioned to a second forecasting data model without incurring downtime in the computing environment. After the transition, the usage of the computing resources of the computing environment is forecasted using the second forecasting data model and the usage measurements obtained from the computing resources. The second forecasting data model exponentially decays the usage measurements based on a respective time period at which the usage measurements were obtained.

Exponential decay real-time capacity planning

Computational methods and systems for detecting and troubleshooting anomalous behavior in distributed applications executing in a distributed computing system are described herein. Methods and systems discover nodes comprising the application. Anomaly detection monitors the metrics associated with the nodes for anomalous behavior in order to identify an approximate point in time when anomalous behavior begins to adversely impact performance of the application. Anomaly detection also monitors logs messages associated with the nodes to detect anomalous behavior recorded in the log messages. When anomalous behavior is detected in either the metrics and/or the log messages an alert identifying the anomalous behavior is generated. Troubleshooting guides an administrator and/or application owner to investigate the root cause of the anomalous behavior. Appropriate remedial measures may be determined based on the root cause and automatically or manually executed to correct the problem.

Methods and systems for troubleshooting applications using streaming anomaly detection

Automated methods and systems described herein are directed to identifying potential root causes of a problem in a data center. Methods and systems receipt an alert or other notification of a problem occurring in a data center and a time when the problem was noticed. A search window is created based on the time and a stream of log messages generated in the search window is converted into a time dependent metric. An anomaly detection technique is applied to the metric to determine a start time of a problem. Logging events and key phrases in the log messages are identified in the search window and presented as potential root causes of the problem. The potential root cause may then be used by system administrators and/or tenants to diagnose the problem and execute remedial measures to correct the problem.

Methods and systems for determining potential root causes of problems in a data center using log streams

Various examples are disclosed for forecasting resource usage and computing capacity utilizing an exponential decay. In some examples, a computing environment can obtain usage measurements from a data stream over a time interval, where the usage measurements describe utilization of computing resource. The computing environment can generate a weight function for individual ones of the usage measurements, where the weight function exponentially decays the usage measurements based on a respective time period at which the usage measurements were obtained. The computing environment can forecast a future capacity of the computing resources based on the usage measurements and the weight function assigned to the individual ones of the usage measurements. The computing environment can further upgrade a forecast engine to use the exponential decay without resetting the forecast engine or its memory.

Various examples are disclosed for workload placement using forecast data. Forecast data for workloads and providers during a predefined period of time in the future is considered when identifying stressed providers and the feasibility of a workload move. Workloads with demand spikes at different future times can be matched by stacking current demand and forecast demand by timestamps. The possibility of stress can be reduced by making moves preemptively and considering forecast demand when evaluating the feasibility of a workload move.

Workload placement with forecast

Computational methods and systems that proactively manage usage of computational resources of a distributed computing system are described. A sequence of metric data representing usage of a resource is detrended to obtain a sequence of non-trendy metric data. Stochastic process models, a pulse wave model and a seasonal model of the sequence of non-trendy metric data are computed. When a forecast request is received, a sequence of forecasted metric data is computed over a forecast interval based on the estimated trend and one of the pulse wave or seasonal model that matches the periodicity of the sequence of non-trendy metric data. Alternatively, the sequence of forecasted metric data is computed based on the estimated trend and the stochastic process model with a smallest accumulated residual error. Usage of the resource by virtual objects of the distributed computing system may be adjusted based on the sequence of forecasted metric data.

Methods and systems to proactively manage usage of computational resources of a distributed computing system

The current document is directed to methods and systems that collect metric data within computing facilities, including large data centers and cloud-computing facilities. In a described implementation, input metric data is compressed by replacing each metric data point with a one-bit, two-bit, four-bit, or eight-bit compressed data value. During a first time window following reception of a metric data point, the metric data point remains available in uncompressed form to facilitate data analysis and monitoring functionalities that use uncompressed metric data. During a second time window, the metric data point is compressed and stored in memory, where the compressed data point remains available for data analysis and monitoring functionalities that use compressed metric data for detection of peaks, periodic patterns, and other characteristics. Finally, the compressed data point is archived in mass storage, where it remains available to data-analysis and management functionalities for a lengthy time period.

Methods and systems that efficiently store metric data to enable period and peak detection

A method for processing queries for a document of elements is provided. The document includes a plurality of subsections where each subsection includes at least a portion of elements in the document. The method comprises: receiving a query for a path of elements in the document of elements; determining a plurality of step queries from the query, each step query including at least a part of the path of elements; for each step query in the plurality of step queries, determining one or more subsections that include elements that correspond to a step query; and determining at least one subsection that includes the path of elements of the query. A result for the query is generated using the at least one subsection.

Parent-child query indexing for XML databases

Parent-child query indexing for xml databases

A method for processing queries for a document of elements is provided. The document includes a plurality of subsections where each subsection includes at least a portion of elements in the document. The method comprises: receiving a query for a npath of elements in the document of elements; determining a plurality of step queries from the query, each step query including at least a part of the path of elements; for each step query in the plurality of step queries, determining one or more subsections that include elements that correspond to a step query; and determining at least one subsection that includes the path of elements of the query. A result for the query is generated using the at least one subsection.

One aspect of the present invention is a system for classifying element nodes in a subtree-structured XML database. The XQE structural-textual classification system is sensitive to both the textual resemblance between document elements as well as the structural resemblance between document elements. The XQE structural-textual classification system might use the XQE parent-child index described in Lindblad II-A for the purpose of forming vectors of 'terms' which encode both the structural and the textual content of XML elements. The element vectors are processed by a classifier to create class prototype vectors which can be used to classify elements as they are added to the database.

Growing community of inventors

Palo Alto, CA, United States of America

Paul Pedersen