Splunk Inc.

Cribl, Inc.

Clint Sharp

Ledion Bitincka

Tristan Antonio Fletcher

Dritan Bitincka

Hemendra Singh Choudhary

Alexander Douglas James

Fang I Hsiao

David Marquardt

Vladimir A Shcherbakov

Michael R Dickey

Thomas Allan Haggie

Clayton S Ching

Nicholas Robert Romito

Oliver Draese

Nima Haddadkaveh

Anupadmaja Raghavan

Elizabeth Lin

Jason Szeto

Petter Eriksson

Steve Yu Zhang

Alok Anant Bhide

Christopher Madden Pride

Vishal Patel

David Everett Simmen

Igor Stojanovski

Leonid Alekseyev

Brent Stephen Boe

Hailun Yan

Donn Rochette

Murugan Kandaswamy

Marc Andre Chene

Amrittpal Singh Bath

Pratiksha Shah

Jesse Miller

Esguerra Ma Kharisma

John Chelikowsky

A search support system allows a customer to browse data contained in files stored on an external storage system. The search support system allows a customer to specify data processing tasks to be performed on raw data retrieved from a file stored on the external storage system. The customer specifies each data processing task and the search support system performs each task as it is selected by the customer on raw data retrieved from the file. The search support system concurrently displays the results of each data processing task in real time in a graphical user interface. The search support system saves the customer's settings as a late binding schema that can be applied to raw data retrieved from the external storage system in order to parse the raw data and to create, index, and search timestamped events derived from the raw data.

File browser user interface

In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a method of searching remotely-stored data includes receiving a search query at a computer node residing at a node geolocation, the search query representing a request to search data stored at a storage geolocation; identifying compute geolocations each including computing resources, each of the compute geolocations being distinct from the node geolocation and the storage geolocation; obtaining latency data including latency values for the compute geolocations; selecting one of the compute geolocations based on the latency data; by operation of a coordinator agent on the computer node, initiating a dynamic computing resource at the selected compute geolocation, and receiving search results identified by the dynamic computing resource.

Searching remote data in an observability pipeline system

In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search query is received at a computer node from a leader role of an observability pipeline system. The search query represents a request to search event data at the computer node and includes a first search operator that specifies a system state context criterion and a second search operator that specifies an event criterion. An observability pipeline process is configured according to the search query. Search results are generated based on applying the observability pipeline process at the computer node. A current system state of the computer node that matches the first search operator is determined; and a subset of event data on the computer node that matches the second search operator is identified. The search results including the subset of event data are sent to the leader role.

Targeting system state context in a search process in an observability pipeline system

In some aspects, search functionality is provided in an observability pipeline system. In some implementations, a search method includes receiving a search query from a leader role in an observability pipeline system. The search query represents a request to search event data at a computer resource. An observability pipeline process is configured to perform a search according to the search query, and search results are obtained based on applying the observability pipeline process to the event data. The search results include events from the event data. Provenance information is obtained for each of the events. The provenance information for each event includes an identification of the computer resource and a link to the computer resource. Augmented search results are generated by associating the provenance information with the respective events, and the augmented search results are communicated to the leader role.

Enriching search results with provenance information in an observability pipeline system

Data payloads from an external data storage system are processed in an observability pipeline system. In some aspects, the observability pipeline system defines a leader role and worker roles. The leader role generates a data discovery task based on configuration information for a data collection task. A worker role executes the data discovery task, which includes communicating with an external data storage system to identify a data payload that is stored on the external data storage system and contains event data that meet event filter criteria. The leader role generates data collection tasks based on the data payload. Worker roles execute the data collection tasks. Executing a data collection task includes: communicating with the external data storage system to obtain a subset of filtered event data from the data payload; and streaming the subset of filtered event data to an observability pipeline process.

Processing data payloads from external data storage systems in an observability pipeline system

Load balancing processes are performed in an observability pipeline system comprising a plurality of computing resources. In some aspects, the observability pipeline system defines a leader role and worker roles. A plurality of computing jobs each include computing tasks associated with event data. The leader role dispatches the computing tasks to the worker roles according to a least in-flight task dispatch criteria, which includes iteratively: identifying an available worker role; identifying one or more incomplete computing jobs; selecting, from the one or more incomplete computing jobs, a computing job that has the least number of in-flight computing tasks currently being executed in the observability pipeline system; identifying a next computing task from the selected computing job; and dispatching the next computing task to the available worker role. The worker roles execute the computing tasks by applying an observability pipeline process to the event data associated with the respective computing task.

Load balancing computing resources in an observability pipeline system

In some aspects, data collection functions are interposed to generate input data for an observability pipeline system. In some aspects, a data collection function is made available to an application running on a computer system, with the data collection function having the same name as an original function referenced by the application. In response to a call to the original function, the data collection function is executed and data is extracted from the application. The original function is then executed. A reporting thread of the application is executed; executing the reporting thread generates observability pipeline input data by formatting the extracted data and sends the observability pipeline input data from the computer system to an observability pipeline system.

Function interposition in an observability pipeline system

The disclosed embodiments provide a system for extracting custom content from network packets. During operation, the system receives a stream of packets. The system then parses packets in the stream to determine a protocol for each packet. Next, the system applies a custom-content-extraction rule to each packet associated with a target protocol to obtain the extracted content. Then, the system stores the extracted content in events in a data store to facilitate subsequent queries involving the extracted content.

Generating event data at remote capture agents based on identified network addresses

Technologies are disclosed for providing a common information model. Features include: detecting a scheduled time for a key performance indicator reflecting how a service provided by one or more entities is performing, entity definition information recording the association between the entities and its machine data, service definition information associating the entities that provide the service, and the KPI being defined by a search query, including a field identifier specified in a data model, the KPI derives a value from the machine data; performing the query in response to said detecting, including: associating values in the machine data having disparate field names in accordance with disparate schemas with the field identifier specified in the data model, and processing the associated values as semantically equivalent data instances. In doing so, values having the same semantic (or related semantics) can be used together despite being associated with disparate field names from disparate schemas.

Evaluating key performance indicators of information technology service

The disclosed embodiments include a method performed by a data intake and query system. The method includes populating each metric including a measure value, cataloging metadata in an in-memory metrics catalog, where the metadata is related to the metrics. The method further includes receiving a search query including search criteria, evaluating the search query by applying the search criteria to the metadata of the metrics catalog to obtain results that satisfy the search criteria, and causing display, on a display device, of the results or data indicative of the results.

In-memory catalog for searching metrics data

Techniques are disclosed for providing an aggregate key performance indicator (KPI) that spans multiple services and is based on a priority value. The techniques may enable a user to select KPIs and to adjust weights (e.g., importance) associated with the KPIs. The weight of a KPI may affect the influence a value of the KPI has on the calculation of an aggregate KPI value. The techniques may also include the ability to create a correlation search using the selected KPIs and weights so that a notification may be generated when the aggregate KPI value exceeds a threshold.

Aggregate key performance indicator spanning multiple services and based on a priority value

Performing search queries for key performance indicators using an optimized common information model

The disclosed embodiments include a method performed by a data intake and query system. The method includes ingesting each metric including at least one key value and a measured value taken of a computing resource, and storing each metric in an index of a metrics store, where the index defines at least one dimension populated with the at least one key value and a measure populated with the measured value. The method further includes cataloging metadata in a metrics catalog, where the metadata is related to the metrics stored in the metrics store, performing an analysis of metrics data included in the metrics store and/or the metrics catalog to obtain results, and causing display of the results or an indication of the results on a display device.

Metrics store system

Facilitating custom content extraction rule configuration for remote capture agents

Growing community of inventors

Oakland, CA, United States of America

Clint Sharp