Vmware, Inc.

Jobin Raju George

Kumar Gaurav

Chandrashekhar Jha

Ritesh Ranjan Jha

Nikhil Jaiswal

Jusvinder Singh

Pushkar Patil

Vaidic Joshi

Shivam Satija

Prateek Sahu

Aditya Gopisetti

Gurusreekanth Chagalakondu

Ajay Gautam

Naveen Mudnal

Jayalekshmy Nair

Shivanju Awasthi

The current document is directed to methods and systems that automatically identify log/event-message traces and computational-entity transactions within collections and/or streams of log/event messages. Automated identification of log/event-message traces provides the basis for automated interpretation, by automated computer-system administration-and-the management subsystems, of the information represented by collections and/or streams of log/event messages. Disclosed approaches to automatically identifying log/event-message traces and computational-entity involve identifying log/event-message types, generating time-series-like log/event-message-type occurrence signals from log/event-message collections and/or streams, and computing cross correlations between pairs of log/event-message-type occurrence signals. In one implementation, a strongly-correlated-type graph is generated from the computed cross correlations, from which connected-components subgraphs, corresponding to computational-entity transactions, are extracted. Log/event-message traces are then extracted from acyclic graphs generated from the connected-component subgraphs.

Methods and systems that identify computational-entity transactions and corresponding log/event-message traces from streams and/or collections of log/event messages

The detection of utilized virtual machines through usage pattern analysis is described. In one example, a computing device can collect utilization metrics from a virtual machine over time. The utilization metrics can be related to one or more processing usage, disk usage, network usage, and memory usage metrics, among others. The utilization metrics can be used to determine a number of clusters, and the clusters can be used to organize the utilization metrics into groups. Depending upon the number or overall percentage of the utilization metrics assigned to individual ones of the plurality of clusters, it is possible to determine whether or not the virtual machine is a utilized or an idle virtual machine. Once identified, utilized virtual machines can be migrated in some cases. Idle virtual machines can be shut down to conserve processing resources and costs in some cases.

Usage pattern virtual machine idle detection

The current document is directed to methods and systems that efficiently process and store log/event messages generated within distributed computer facilities. Various different types of initial processing steps may be applied to a stream of log/event messages received by a message-collector system or a message-ingestion-and-processing subsystem. The currently disclosed methods and systems employ additional pre-processing steps to identify the types of received log/event messages, monitor event-type-associated log/event-message-usage-delay histories, and employ time-series-analysis-based and/or neural-network-based estimation of event-type-associated log/event-message usage to efficiently store log/event-messages in low-cost and low-latency storage facilities.

Method and subsystem within a distributed log-analytics system that automatically determines and enforces log-retention periods for received log-event messages

The current document is directed to methods and systems that efficiently process log/event messages within and among distributed computer facilities. Various different types of initial processing steps may be applied to a stream of log/event messages received by a message-collector system or a message-ingestion-and-processing system. By including a pre-processing step two identify the type of a received log/event message, and by specifying initial-processing-step criteria with respect to log/event-message types, significant increases in the efficiency of log/event-message preprocessing by message-collector systems and message-ingestion-and-processing systems is achieved.

Efficient event-type-based log/event-message processing in a distributed log-analytics system

The current document is directed to methods and systems that sample log/event messages for downstream processing by log/event-message systems incorporated within distributed computer facilities. The data-collection, data-storage, and data-querying functionalities of log/event-message systems provide a basis for distributed log-analytics systems which, in turn, provide a basis for automated and semi-automated system-administration-and-management systems. By sampling log/event-messages, rather than processing and storing every log/event-message generated within a distributed computer system, a log/event-message system significantly decreases data-storage-capacity, computational-bandwidth, and networking-bandwidth overheads involved in processing and retaining large numbers of log/event messages that do not provide sufficient useful information to justify these costs. Increase in efficiencies of log/event-message systems obtained by sampling translate directly into increases in bandwidths of distributed computer systems, in general, and to increases in time periods during which useful log/event messages can be stored.

Methods and systems that sample log/event messages in a distributed log-analytics system

Methods and systems predict parameters in a dataset of an identified piece of ('information technology') IT equipment. An automated method identifies datasets IT equipment in a same category of IT equipment as a piece of IT equipment identified as having incomplete dataset information. Each dataset of IT equipment parameters is used to construct generalized linear models of different classes of IT equipment within the category of IT equipment. The class of the identified IT equipment is determined. A predicted equipment parameter of incomplete information of the identified piece of IT equipment is computed using the generalized linear model associated with the class. The predicted equipment parameter can be used to complete the dataset of the identified piece of IT equipment.

Methods and systems to predict parameters in a database of information technology equipment

The current document is directed to methods and systems that efficiently cache log/event messages in log/event-message systems incorporated within distributed computer facilities. The log/event-message systems provide the data-collection, data-storage, and data-querying functionalities that provide a basis for distributed log-analytics systems which, in turn, provide a basis for automated and semi-automated system-administration-and-management systems. Efficient caching significantly extends the period of time that networking failures and log/event-message processing delays can be tolerated by log/event-message systems without significant loss of information. Rather than caching individual log/event messages, message collectors that employ the currently disclosed technologies cache information regarding log/event-message types, leading to efficient compression of the cached information. As a result, message collectors can cache log/event messages without significant loss of information for time periods of several orders of magnitude greater than those achieved by caching individual log/event messages.

Methods and systems that efficiently cache log/event messages in a distributed log-analytics system

Methods and systems that automatically rank log/event messages and log/event-message transactions to facilitate analysis of log/event-messages generated within distributed-computer systems are disclosed. A base-window dataset and current-window dataset are selected for diagnosis of a particular error or failure and processed to generate a transaction sequence for each dataset corresponding to log/event-message traces identified in the datasets. Then, frequencies of occurrence of log/event-message types relative to transaction types are generated for each dataset. From these two sets of relative frequencies of occurrence, changes in the relative frequency of occurrence for each log/event-message-type/transaction-type pair are generated. Normalized scores for log/event-message-type/transaction-type pairs and scores for transaction types are then generated from the changes in the relative frequency of occurrence. The generated scores reflect the relevance of log/event-messages in traces corresponding to particular transaction as well as the relevance of transaction types to the error or failure.

Methods and systems that rank and display log/event messages and transactions

The detection of idle virtual machines through usage pattern analysis is described. In one example, a computing device can collect utilization metrics from a virtual machine over time. The utilization metrics can be related to one or more processing usage, disk usage, network usage, and memory usage metrics, among others. The utilization metrics can be separated into a set of training metrics and a set of validation metrics, and a number of clusters can be determined based on the set of training metrics. The clusters can be used to organize the set of validation metrics into groups. Depending upon the number or overall percentage of the utilization metrics assigned to individual ones of the plurality of clusters, it is possible to determine whether or not the virtual machine is an idle virtual machine. Once identified, idle virtual machines can be shut down to conserve processing resources and costs.

Various examples are disclosed for using clustering routines to extrapolate metrics to other computing resources in a cluster. One or more computing devices can classify computing resources, such as servers, based on various characteristics of the computing resources. For each class of computing resource, a clustering routine can be applied to generate clusters of the computing resources. A minimal number of metrics required to be obtained from an end user can be determined as a function of a number of the clusters. If one or more of the metrics are obtained from the end user, the metrics can be extrapolated to other computing resources in the same cluster.

Clustering routines for extrapolating computing resource metrics

System and computer-implemented method for managing workloads in a computing environment use collected resource utilization metrics for a workload of a compute instance running in the computing environment that are superimposed to produce a unified time series of the resource utilization metrics for the workload. Undesired periodic patterns are then removed from the unified time series of the resource utilization metrics, which is used to identify the compute instance as a particular type of compute instances. Based on the identification, an action is recommended and implemented on the compute instance.

System and method for managing workloads using superimposition of resource utilization metrics

Methods and systems predict parameters in a dataset of an identified piece of ('information technology') IT equipment. An automated method identifies datasets IT equipment in a same category of IT equipment as a piece of IT equipment identified as having incomplete dataset information. Each dataset of IT equipment parameters are used to construct generalized linear models of different classes of IT equipment within the category of IT equipment. The class of the identified IT equipment is determined. A predicted equipment parameter of incomplete information of the identified piece of IT equipment is computed using the generalized linear model associated with the class. The predicted equipment parameter can be used to complete the dataset of the identified piece of IT equipment.

Methods and systems are directed to detection and correction of outliers in a dataset stored in a data-storage device. The dataset comprises parameter data that may be stored and organized in the form of a data table with rows and columns of parameter values. Each column of the parameter data is searched for outlier parameter values based on the parameters values in the same column. The parameter data as a whole may be searched for outlier rows of parameter values based on first and second largest variations in the parameter data. Substitute parameter values are determined for the outlier parameter values based on non-outlier parameter values of the parameter data. The substitute parameter values and corresponding outlier parameter values may be displayed in a database management user interface that enables a user to selectively accept or reject each of the substitute parameter values for the corresponding outlier parameter values.

Methods and systems to detect and correct outliers in a dataset stored in a data-storage device

Techniques for migrating a VM in a hybrid cloud computing system are provided. The techniques include identifying a similar VM at the destination, comparing dictionaries for the VM to be transmitted and the similar VM, and compressing the VM based on the comparison. After transmitting the compressed VM, the destination decompresses the VM using the dictionary of the similar VM. Dictionaries associate chunks of VM data with hashes of those chunks. This allows replacement of chunks with the hashes, thereby compressing the VM for transmission.

Growing community of inventors

Bengaluru, India

Jobin Raju George