Splunk Inc.

Splunkinc.

Steve Yu Zhang

Stephen Phillip Sorkin

Ledion Bitincka

David Marquardt

Archana Sulochana Ganapathi

Marc Vincent Robichaud

Alice Emily Neels

Igor Stojanovski

Adam Jamison Oliner

Karthikeyan Sabhanatarajan

Vishal Patel

Jesse Brandau Miller

Alexandros Batsakis

Jacob Barton Leverich

Jindrich Dinga

Katherine Kyle Feeney

Kristal Curtis

Nicholas Robert Romito

Yuan Xie

Balaji Rao

Jianming Zhang

Alexander Douglas James

David Everett Simmen

Erick Anthony Dean

Jiahan Wang

Pradeep Baliganapalli Nagaraju

Nitilaksha Halakatti

Jay A Pathak

Prem Kumar Jayaraj

Christopher Madden Pride

Anish Shrigondekar

Thomas Allan Haggie

Clint Sharp

Arun Ramani

Ruochen Zhang

Hailun Yan

Murugan Kandaswamy

Sophia Rui Zhu

Marc Andre Chene

Nir Frenkel

Amrittpal Singh Bath

Pratiksha Shah

Manuel Martinez

Ningxuan He

Shyam Mundhra

Esguerra Ma Kharisma

Thor Taylor

Manikandan Vellore Muneeswaran

Phil Yonghui Wang

Nitilaksha Satyaveera Halakatti

Manuel Gregorio Martinez

A computing device can receive a query that identifies a set of data to be processed and determine that a portion of the set of data resides in an external data system. The query system can request data identifiers associated with data objects of the set of data from the external data system and communicate the data identifiers to a data queue. The computing device can instruct one or more search nodes to retrieve the identifiers from the data queue. The search nodes can use the data identifiers to retrieve data objects from the external data system and process the data objects according to instructions received from the computing device. The search nodes can provide results of the processing to the computing device.

Retrieving data identifiers from queue for search of external data system

Systems and methods include causing presentation of a first cluster in association with an event of the first cluster, the first cluster from a first set of clusters of events. Each event includes a time stamp and event data. Based on the presentation of the first cluster, an extraction rule corresponding to the event of the first cluster is received from a user. Similarities in the event data between the events are determined based on the received extraction rule. The events are grouped into a second set of clusters based on the determined similarities. Presentation is caused of a second cluster in association with an event of the second cluster, where the second cluster is from the second set of clusters.

Supplementing extraction rules based on event clustering

A data intake and query system can generate local data enrichment objects and receive federated data enrichment objects from another data intake and query system. In response to receiving a query, the data intake and query system can determine whether the query is subquery of a federated query. If the query is a subquery, the data intake and query system can use the federated data enrichment objects to execute the query.

Federated data enrichment objects

Embodiments are directed are towards a method for generating a query response, which comprises creating two or more partitions of event records from raw data stored in a data store, wherein each event record in the two or more partitions of event records includes a portion of the raw data and is associated with a time stamp derived from the raw data. The method also comprises generating a summarization table for each partition of the two or more partitions that: (a) identifies a field value comprising a value that corresponds to an associated field extracted from a respective event record; and (b) for the field value, includes a posting value to the respective event record within a respective partition. The method further comprises generating partial results for a received query using summarization tables in the partitions and generating a response to the query by combining the partial results.

Generating a query response by combining partial results from separate partitions of event records

A computer-implemented method of determining indexed fields at query time comprises mapping data from a first source type to indexed fields in batch form using a wildcard specifier. The method also comprises receiving a query to execute on a data set comprising data from the first source type and data from a second source type. Further, the method comprises transforming the query to execute on the data from the first source type separately from the data from the second source type. Additionally, the method comprises executing the query to operate on the data from the first source type using information associated with the indexed fields and to separately operate on the data from the second source type.

Intelligent search-time determination and usage of fields extracted at index-time

Embodiments of the present disclosure provide techniques for using an inverted index in a pipelined search query. A field searchable data store is provided that comprises a plurality of event records, each event record comprising a time-stamped portion of raw machine data. Responsive to the reciept of an incoming search query, the search engine accesses an inverted index, wherein each entry in the inverted index comprises at least one field name, a corresponding at least one field value and a reference value associated with each field name and value pair that identifies a location in the data store where an associated event record is stored. Once the inverted index is accessed, it can be used to identify and search a subset of the plurality of event records, wherein the subset comprises one or more event records with corresponding reference values in the inverted index.

Pipelined search query, leveraging reference values of an inverted index to access a set of event data and performing further queries on associated raw data

Embodiments of the present disclosure provide techniques for performing searches of event records by leveraging reference values in an inverted index. A method of searching comprises accessing a query associated with a first set of event records in a field searchable data store, each event record comprising a time-stamped portion of raw machine data. The method further comprises evaluating the query and generating results for the query by accessing an inverted index, wherein each entry in the inverted index comprises at least one field, a corresponding at least one field value and a reference value that identifies a location in the field searchable data store where an associated event record is stored. The method further comprises performing a search to filter out a second set of event records and retrieving the second set of event records from the field searchable data store using reference values in the inverted index.

Retrieving event records from a field searchable data store using references values in inverted indexes

A method and system for managing searches of a data set that is partitioned based on a plurality of events. A structure of a search query may be analyzed to determine if logical computational actions performed on the data set is reducible. Data in each partition is analyzed to determine if at least a portion of the data in the partition is reducible. In response to a subsequent or reoccurring search request, intermediate summaries of reducible data and reducible search computations may be aggregated for each partition. Next, a search result may be generated based on at least one of the aggregated intermediate summaries, the aggregated reducible search computations, and a query of adhoc non-reducible data arranged in at least one of the plurality of partitions for the data set.

Generating search results based on intermediate summaries

Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.

Data model selection and application based on data sources

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a 'divide and conquer' algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Tracking event records across multiple search sessions

Embodiments of the present invention are directed to identifying related data, in particular, data associated with different source types. In embodiments, a first source type related to a second source type associated with a search query is identified. Field set pairs are identified from a first data set associated with the first source type and a second data set associated with the second source type. Each field set pair can include one field set associated with the first source type and another field set associated with the second source type. For each field set pair, an extent of similarity is determined between the corresponding field sets. Based on the extent of similarities between the corresponding field sets, at least one pair of related field sets is identified. An indication of the at least one pair of related field sets is provided, for example, for presentation to a user.

Identifying related field sets based on related source types

A computer-implemented method of determining indexed fields at query time comprises indexing time-stamped events ingested from a plurality of source types. The time-stamped searchable events compare portions of raw data. The method also comprises generating an index containing each keyword in the time-stamped searchable events and an associated location reference of a respective event in which the keyword appears. Further, the method comprises generating a fields metadata file identifying indexed fields in the time-stamped searchable events for each source type. The fields metadata file comprises reference values for accessing indexed fields associated with each source type from the index. The method also comprises accessing the fields metadata file to identify the indexed fields associated with each source type prior to executing a query.

Storing indexed fields per source type as metadata at the bucket level to facilitate search-time field learning

A data intake and query system executes a search query at a first execution time for querying events having associated time stamps within a first time period characterized by a first start time and a first end time. The first start time is computed based upon a time indicated by reference time information stored prior to execution of the search query. The system determines whether execution of the search query completed successfully based upon a first search result obtained from executing the search query. If the first execution of the search query was not successful, the system computes for a second execution of the search query after the first execution, a second time period using the reference time information. The second execution is configured to query events with associated timestamps that fall within a second time period that includes the first time period and an additional time period.

Durable search queries for reliable distributed data retrieval

Systems and methods for rule-based data stream processing by data collection, indexing, and visualization systems. An example method includes: receiving, by the computer system, an input data stream comprising raw machine data; processing the raw machine data by a data processing pipeline that produces transformed machine data, wherein the data processing pipeline comprises an ordered plurality of pipeline stages, wherein a pipeline stage of the ordered plurality of pipeline stages applies a rule of a set of rules to an input of the pipeline stage, wherein the rule specifies an action to be performed on the input of the pipeline stage responsive to evaluating a conditional expression applied to the input of the pipeline stage, wherein the action generates an output of the pipeline stage, and wherein the rule is selected based on a source type associated with the input data stream; and supplying the transformed machine data to a data collection, indexing, and visualization system.

Rule-based data stream processing

Growing community of inventors

San Francisco, CA, United States of America

Steve Yu Zhang