Splunk Inc.

Microsoft Technology Licensing, LLC

Cisco Technology, Inc.

Alexander Douglas James

Jesse Brandau Miller

Sourav Pal

Christopher Madden Pride

Marc Vincent Robichaud

Cory Eugene Burke

Jeffrey Thomas Lloyd

Arindam Bhattacharjee

Arun Ramani

Andrew Peters

Manu Jose

Scott Calvert

David Marquardt

Bei Li

Alexandros Batsakis

Eric Sammer

Joseph Gabriel Echeverria

Ashish Mathew

Thomas Allan Haggie

Clint Sharp

Xiaowei Wang

Nicholas Robert Romito

James Monschke

Karthikeyan Sabhanatarajan

Andrew E Robbins

Ankit Jain

Ganesh Jothikumar

Igor Braylovskiy

Sogol Moshtaghi

Steve Yu Zhang

Venkatasubramanian Jayaraman

Salih Ammar Wajih Zainulabdeen

Amy Joanna Sutedja

Ankur Dalsukhbhai Bambharoliya

Vinayak Bhakta

Henricus Johannes Maria Meijer

Balan Sethu Raman

Ledion Bitincka

Vishal Patel

Jason Duane Clark

David Everett Simmen

Igor Stojanovski

Lance Olson

Michael Pizzo

Srikanth Mandadi

Siddharth Jayadevan

Bharath Kishore Reddy Aleti

Pablo Castro

Christopher L Anderson

Sai Krishna Sajja

Colin Joseph Meek

Bart J F De Smet

Arthur Foelsche

Ricky Gene Burnett

Hailun Yan

Michael Justin Flasko

Daniel Ferstay

Murugan Kandaswamy

Diego Bernardo Vega

Marc Andre Chene

Pratiksha Shah

Simon Cavanagh

Amrittpal Singh Bath

Kishore Reddy Ramasayam

Rafael De Jesús Fernández Moctezuma

Esguerra Ma Kharisma

Shyam Mundhra

Jengie Shau

David Ryan Marquardt

Thor Taylor

Manikandan Vellore Muneeswaran

Colin James Meek

Operational machine components of an information technology (IT) or other microprocessor- or microcontroller-permeated environment generate disparate forms of machine data. Network connections are established between these components and processors of data intake and query system (DIQS). The DIQS conducts network transactions on a periodic and/or continuous basis with the machine components to receive disparate data and ingest certain of the data as entries of a data store that is searchable for DIQS query processing. The DIQS may receive queries to process against the received and ingested data via an exposed network interface. In one example embodiment, the DIQS receives a query identifying data to be processed, dynamically generates a query processing scheme based on the state of the data to be processed, such as streaming or at rest, and dynamically communicates the query processing scheme to a query executor based on the state of the data to be processed.

Dynamic query processor for streaming and batch queries

Systems and methods are disclosed for receiving, at query coordinator, a search query. The query coordinator parses the search query and generates tasks for different runtime systems. The query coordinator configures an interface enabling inter system communication between the runtime systems. The generated tasks are distributed to the runtime systems and partial results of a runtime system are communicated to the interface. The query coordinator retrieves the partial results from the interface, finalizes the partial results, and sends finalized results them to the requestor of the search query.

Runtime systems query coordinator

Systems and methods are disclosed for generating a distributed execution model with untrusted commands. The system can receive a query, and process the query to identify the untrusted commands. The system can use data associated with the untrusted command to identify one or more files associated with the untrusted command. Based on the files, the system can generate a data structure and include one or more identifiers associated with the data structure in the distributed execution model. The system can distribute the distributed execution model to one or more nodes in a distributed computing environment for execution.

Executing commands from a distributed execution model

Systems and methods are described for a query conversion system to convert a first query string from a first version of a query language to a second version of the query language. The query conversion system may be associated with a tokenizer and parser, code converter, compatibility library, and a query formatter. The tokenizer and parser may tokenize and parse a query string to create a first node tree with commands. The code converter may parse the first node tree while using the compatibility library to convert the commands and generate a second node tree. The query formatter may create a second query string executable by the second version of the query language.

Language conversion system

A method includes displaying events that correspond to search results of a search query, the events comprising data items of event attributes, the events displayed in a table. The table includes columns corresponding to an event attribute, rows corresponding events, cells populated data items, and interactive regions corresponding to at least one data item and selectable to add one or more commands to the search query. A reference event attribute is determined based on an analysis of a data object. A supplemental column corresponding to a supplemental event attribute is added to the table based on the reference event attribute. Supplemental interactive regions are added to the table and correspond to supplemental data items.

Providing supplemental values for events

Systems and methods are disclosed for processing and executing queries against one or more dataset sources, where the queries identify a set of data to be processed and a manner of processing the set of data. To query the dataset sources, a query coordinator generates a query processing scheme that includes a dynamic allocation of multiple layers of partitions. The query is then executed based on the query processing scheme.

Dynamic partition allocation for query execution

A method includes receiving an initial pipeline including a sequence of commands for execution on a computing system, and obtaining, for each command in the sequence of commands, semantic information. The sequence of commands includes a command with incomplete semantic information. The method further includes generating an abstract semantic tree (AST) with the semantic information and a placeholder for the incomplete semantic information, and manipulating the AST to generate a revised AST. The revised AST corresponds to a revised pipeline that reduces an execution time on the computing system. The method further includes executing the revised pipeline.

Partially typed semantic based query execution optimization

Systems and methods are described for implementing programmatic input/output (I/O) routing to datasets with user-defined partitions while providing unhandled data protection. As disclosed herein, a user may define a dataset as including one or more partitions, each partition including criteria for storing data objects written to the partitioned dataset in the individual partitions. Data objects written to the dataset can then be evaluated according to the criteria, and routed to an appropriate partition. To provide unhandled data protection, a dataset definition can include a default partition to which data objects are routed when the data object fails to satisfy the criteria of any of the set of user-defined partitions identified in the specification. Processing I/O operations according to a user-defined partitioning schema can enable data objects to be arranged according to any partitioning schema without tethering the partitioning to a particular underlying storage system.

Unhandled data protection for programmatic input/output routing to datasets with user-defined partitions

A method includes assigning an access permission of a first user to a query object that represents a first query, the access permission granting the first user access rights to one or more data sources of the first query the access permission being assigned as a runtime permission of the first query, granting a request from a second user to execute a second query, the first query being a subquery of the second query, and allowing the second user to execute the first query on the one or more data sources of the first query using the runtime permission assigned to the first query in executing the second query using the first query as the subquery.

Query execution using access permissions of queries

Systems and methods are described for providing previews of deployment of data stream processing instructions sets, sometimes called pipelines, to a stream data processing system. Rather than deploying such an instruction set, which may cause detrimental side effects, previews can be facilitated by conversion of a data stream processing instructions set to a batch query that is applied to an existing data set. An output of the batch query can then be provided to an end user as a preview of output of the data stream processing instructions set, when implemented.

Data stream processing instruction set previews using batch conversion

A method includes receiving, in a first query interface, a query composed by the user by typing commands into a query box of the first query interface and based on the receiving of the query, causing events corresponding to query results of the query to be displayed in the first query interface with fields corresponding to the events. Based on the selection by the user of an option, a second query interface is displayed with a table that includes events that correspond to query results of a loaded query. The table includes columns corresponding to event attributes, rows corresponding to events. Cells are populated with the data items of event attributes, where one of the columns corresponds to a field of the fields displayed in the first query interface. The table also includes interactive regions selectable by the user to add one or more commands to the loaded query.

Loading queries across interfaces

In some embodiments, a method may include display of a data summary view of a set of events that correspond to query results of a query. Each event of the set of events may include data items of a plurality of event attributes. In embodiments, the data summary view can include various summary reports. Each summary report can include summary entries and a summary graph that each present a summary of data items of a selected event attribute, of the plurality of event attributes. At least one summary report can include summary entries that are selectable by a user. The method may further include filtering the set of event, in response to, and based on, selection of one or more of the selectable summary entries by the user and updating of at least the first and second summary graphs to correspond to the filtered set of events.

Summarized view of search results with a panel in each column

Systems and methods for rule-based data stream processing by data collection, indexing, and visualization systems. An example method includes: receiving, by the computer system, an input data stream comprising raw machine data; processing the raw machine data by a data processing pipeline that produces transformed machine data, wherein the data processing pipeline comprises an ordered plurality of pipeline stages, wherein a pipeline stage of the ordered plurality of pipeline stages applies a rule of a set of rules to an input of the pipeline stage, wherein the rule specifies an action to be performed on the input of the pipeline stage responsive to evaluating a conditional expression applied to the input of the pipeline stage, wherein the action generates an output of the pipeline stage, and wherein the rule is selected based on a source type associated with the input data stream; and supplying the transformed machine data to a data collection, indexing, and visualization system.

Rule-based data stream processing

Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system parses the query and uses a metadata catalog to dynamically identify configuration parameters of datasets and/or rules associated with the query. The identified configuration parameters are communicated to a query processing component of the data intake and query system for use in executing the query.

Identifying configuration parameters for a query using a metadata catalog

Systems and methods are disclosed for annotating a metadata catalog in a data intake and query system based on a query received by the data intake and query system. The metadata catalog can store information about datasets associated with the data intake and query system, including dataset configuration records of the datasets, which can be used to process queries for execution by the data intake and query system. The data intake and query system can receive a query identifying a set of data to be processed and a manner of processing the set of data. The data intake and query system can parse the query to identify datasets and/or data fields associated with the query. Based on the identified datasets and/or fields, the data intake and query system can generate one or more annotations, and use the annotations to update the metadata catalog.

Revising catalog metadata based on parsing queries

Growing community of inventors

Sammamish, WA, United States of America

Alexander Douglas James