Splunk Inc.

Hewlett-packard Development Company, L.p.

Splunkinc.

Stephen Phillip Sorkin

Steve Yu Zhang

Robin Kumar Das

Andre David Stechert

Michael Joseph Baum

Erik M Swan

Rory Greene

Brian Philip Murphy

R David Carasso

Nicholas Christian Mealy

Bradley Hall

Mitchell Blank, Jr

Christina Frances Regina Noren

Vishal Patel

David Marquardt

Ledion Bitincka

Marc Vincent Robichaud

Archana Sulochana Ganapathi

Alice Emily Neels

Igor Stojanovski

Amritpal Singh Bath

David Carasso

Micah James Delfino

Eric Timothy Woo

Johnvey Hwang

Leonid Budchenko

Sundar Vasan

Brad Hall

Joshua Rogers Tyler

Rajan Mathew Lukose

Jimmy John

Johnathon Lee Cervelli

Nick Mealy

Bernardo A Huberman

Mark David Lillibridge

Christina Noren

Brian Phillip Murphy

Embodiments are directed are towards a method for generating a query response, which comprises creating two or more partitions of event records from raw data stored in a data store, wherein each event record in the two or more partitions of event records includes a portion of the raw data and is associated with a time stamp derived from the raw data. The method also comprises generating a summarization table for each partition of the two or more partitions that: (a) identifies a field value comprising a value that corresponds to an associated field extracted from a respective event record; and (b) for the field value, includes a posting value to the respective event record within a respective partition. The method further comprises generating partial results for a received query using summarization tables in the partitions and generating a response to the query by combining the partial results.

Generating a query response by combining partial results from separate partitions of event records

Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is stored as discrete events time stamps. A search is received and relevant event information is retrieved based in whole or in part on the time stamp, a keyword indexing mechanism, or statistical indices calculated at the time of the search.

Search phrase processing

Embodiments are directed towards a method for searching data. The method comprises providing an inverted index that comprises at least one record, wherein the at least one record comprises at least one field name and a corresponding at least one field value. The at least one field name and corresponding value are extracted from time-stamped searchable events that are stored in a field searchable datastore and comprise portions of raw data. The at least one record further comprises a posting value that identifies a location in the field searchable datastore where an event associated with the at least one record is stored. The method further comprises evaluating an incoming search query that references a field name. Responsive to the evaluating, the method comprises determining results for the incoming search query by executing the incoming search query across the field searchable datastore or the inverted index.

Query handling using a field searchable datastore or an inverted index

Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.

Facilitating existing item determinations

A method and system for managing searches of a data set that is partitioned based on a plurality of events. A structure of a search query may be analyzed to determine if logical computational actions performed on the data set is reducible. Data in each partition is analyzed to determine if at least a portion of the data in the partition is reducible. In response to a subsequent or reoccurring search request, intermediate summaries of reducible data and reducible search computations may be aggregated for each partition. Next, a search result may be generated based on at least one of the aggregated intermediate summaries, the aggregated reducible search computations, and a query of adhoc non-reducible data arranged in at least one of the plurality of partitions for the data set.

Generating search results based on intermediate summaries

Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.

Data model selection and application based on data sources

Embodiments are directed towards previewing results generated from indexing data raw data before the corresponding index data is added to an index store. Raw data may be received from a preview data source. After an initial set of configuration information may be established, the preview data may be submitted to an index processing pipeline. A previewing application may generate preview results based on the preview index data and the configuration information. The preview results may enable previewing how the data is being processed by the indexing application. If the preview results are not acceptable, the configuration information may be modified. The preview application enables modification of the configuration information until the generated preview results may be acceptable. If the configuration information is acceptable, the preview data may be processed and indexed in one or more index stores.

Facilitating event creation via previews

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a 'divide and conquer' algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Tracking event records across multiple search sessions

Methods and apparatus consistent with the invention provide the ability to organize and build understandings of machine data generated by a variety of information-processing environments. Machine data is a product of information-processing systems (e.g., activity logs, configuration files, messages, database records) and represents the evidence of particular events that have taken place and been recorded in raw data format. In one embodiment, machine data is turned into a machine data web by organizing machine data into events and then linking events together.

Segmenting machine data into events to identify matching events

Query acceleration using intermediate summaries

A method of data replication in a clustered computing environment comprises receiving, at a selected indexer within a plurality of indexers in a cluster, data from a forwarder indexer, wherein the selected indexer is designated as a primary indexer for the data, wherein the primary indexer has primary responsibility for responding to search queries pertaining to the data, wherein the cluster comprises a plurality of sites. The method further comprises receiving, at the selected indexer, data replication instructions, wherein the data replication instructions comprise a number of other indexers in the cluster for storing a replicated copy of the data and further comprise a number of sites from the plurality of sites across which to store a replicated copy of the data determined in accordance with a site replication factor.

Data replication and site replication in a clustered computing environment

Segmenting machine data into events based on source signatures

Monitoring changes to data items using associated metadata

Storing events derived from log data and performing a search on the events and data that is not log data

Time series search phrase processing

Growing community of inventors

San Fracisco, CA, United States of America

Stephen Phillip Sorkin