Vmware, Inc.

Zhenmin Li

Chengdu Huang

Spiros Xanthos

Jay A Patel

Ron Oded Gery

Erik Hinterbichler

Qingbo Zhu

Yuanyuan Zhou

Xiao Ma

Exemplary methods, apparatuses, and systems parse data with a plurality of pattern matching rules. Pattern matching rules are applied in an ordered sequence and a first rule is applied to the data before a second rule is applied to the data. In response to determining the second rule matches the data, the sequence is reordered to apply the second rule prior to the first rule.

Efficient data pattern matching

Large amounts of unstructured or semi-structured log data generated by software and infrastructure components of a computing system are processed to identify anomalies and potential problems within the computing system. Stored log messages may be queried and analyzed according to dynamic fields constructed from the content of the log messages. As time goes on, the dynamic fields may be converted into static fields which are extracted and indexed at the time of ingestion of the log messages.

Dynamically converting search-time fields to ingest-time fields

A log analytics graphical user interface enables a user to dynamically extract and define a field from unstructured log data. The log analytics module automatically determines a definition for a field based on log text selected by the user. A portion of each log message is highlighted to reflect what the extracted field may be to assist users with understanding if input parameters are selected the intended log data. Changes to the definition of the field, by the user, may cause further highlighting that to indicate an incomplete or erroneous field definition.

Dynamic field extraction of data

Techniques for reducing query response time are provided. In one embodiment, a computer system can organize data into a plurality of buckets, where the data is ordered according to a dimension, and where each bucket includes a subset of data that corresponds to a range within the dimension. The computer system can then precompute, for one or more buckets in the plurality of buckets, query results for one or more queries against the bucket, and can store the precomputed query results in a cache.

Predictive query result computation

Generating an inverted index is disclosed. Semi-structured data from a plurality of sources is parsed to extract structure from at least a portion of the semi-structured data. The inverted index is generated using the extracted structure. The inverted index includes a location identifier and a data type identifier for one or more entries of the inverted index.

Indexing and querying semi-structured data

Exemplary methods, apparatuses, and systems receive data as input to be parsed. The data is parsed using a plurality of pattern matching rules, the plurality of pattern matching rules organized according to a hierarchy including a parent rule and one or more child rules of the parent rule. Parsing includes applying the parent rule to the unstructured data, determining the parent rule is unable to find a pattern match in the unstructured data, and bypassing the application of each child rule to the unstructured data in response to the determination that the parent rule is unable to find a pattern match.

Techniques for scheduling query execution are provided. In one embodiment, a computer system can receive a query to be executed and can assign a priority to the query. The computer system can further divide the query into a plurality of sub-queries and can assign a sub-priority to each sub-query, where the sub-priority is based on a resource consumption metric of the query. The computer system can then select, from a plurality of sub-query pools, a sub-query pool that includes sub-queries of queries that have the same priority as the query, and can add the plurality of sub-queries to the selected sub-query pool.

Dynamic priority-based query scheduling

Exemplary methods, apparatuses, and systems for parsing unstructured data with a plurality of pattern matching rules are disclosed. An optimized pattern matching rule for one or more respective pattern matching rules is derived from an original pattern matching rule. The optimized pattern matching rule includes an extracted text string from the respective pattern matching rule or a less complex pattern match than the respective pattern matching rule. If the extracted text string or pattern is determined to match any of the data to be parsed, application of the original pattern matching rule is bypassed. The original pattern matching rule is applied when the one or more optimized pattern matching rules match the data.

Dynamic field extraction of log data

Indexing data is disclosed. A plurality of subunits of data is determined within received data. For a first subunit, a first mapping between the first subunit and a first dictionary entry is determined. For a second subunit, a second mapping between the second subunit and a second dictionary entry is determined. At least the first and second dictionary entries are aggregated into an aggregation and stored in an index. Searching data is also disclosed. A plurality of subunits of data is determined within a received query. For a first subunit, a first mapping is determined between the first subunit and a first dictionary entry. For a second subunit, a second mapping is determined between the second subunit and a second dictionary entry. The first and second dictionary entries are aggregated and used to search an index.

Growing community of inventors

Mountain View, CA, United States of America

Zhenmin Li