Splunk Inc.

Signalfx, Inc.

Cisco Technology, Inc.

Rajesh Raman

Arijit Mukherji

Phillip Liu

Kris Grandy

Jack William Lindamood

Joseph Ari Ross

Steven Karis

Amin Moshgabadi

Ankit Bhagat

Maxime Petazzoni

Houwu Bai

Darshan Patel

Nishant Agarwal

Nasim Bigdelu

Rebecca Tortell

Uday Sagar Shiramshetty

Mirjana Tesic

Margaret Kelley

Jennifer Williamson

Tianyu Wang

Edward Crossman

Jack Lindamood

Philip Liu

Techniques used by an observability system for facilitating near real-time analytics using structures representing or summarizing data distributions, such as histograms. An observability system receives a set of datapoints in which each received datapoint includes data for a structure representing a data distribution. For example, each of the datapoints may include data for a histogram including at least two dimensions. A processing pipeline is disclosed for processing the received data and making the results of the processing available to a user in near real-time. The processing includes identifying, from the datapoints received by the observability system, a subset of datapoints that fall within a time window, generating aggregate data based upon data in the identified subset of datapoints, computing one or more statistical measures based upon the aggregate data, and enabling querying of the subset of datapoints, the aggregate data, and the statistical measures computed for the first minute.

Near real-time analytics using structures representing data distributions

A data analysis system processes data generated by instrumented software. The data analysis system receives data streams generated by instances of instrumented software executing on systems. The data analysis system also receives metadata describing data streams. The data analysis system receives an expression based on the metadata. The data analysis system receives data of data streams for each time interval and computes the result of the expression based on the received data values. The data analysis system repeats these steps for each time interval. The data analysis system may quantize data values of data streams for each time interval by generating an aggregate value for the time interval based on data received for each data stream for that time interval. The data analysis system evaluates the expression using the quantized data for the time interval.

Real-time reporting based on instrumentation of software

Systems and methods are described for generation of a query using a non-textual input. For example, the query can be generated using a point and click input. A selection of a data source can be identified and an initial query can be automatically generated based on the selection of the data source. A graphical user interface can be displayed and populated with one or more selectable parameters based on the initial query. A selection of the one or more selectable parameters can be received as a non-textual input and a query can be automatically generated based on the selection. For example, a query for execution by a data intake and query system can be generated based on the selection. The query can be provided to the data intake and query system. The data intake and query system may then execute the query on a set of data.

Generation of queries using non-textual input

According to embodiments, a data stream including a plurality of time series data is received and metadata objects are extracted from the data stream. The metadata objects are associated with metrics time series (MTS) objects. The metadata objects and MTS objects are stored via separate in-memory data structures in a logical database. The in-memory data structures include information that correlates the metadata objects with the MTS objects. Any updates to the metadata objects will stay with the metadata objects and do not propagate to the MTS objects. A logical in-memory join may be performed to associate the metadata objects with the appropriate MTS object according to the in-memory data structures when a query for an MTS object is received.

Optimized storage of metadata separate from time series data

Systems and methods are described for generation and execution of modified queries. An input can be received via a visualization of a user interface. The input may identify a first field value and a first field for execution of a query. A set of data for execution of the query can be identified based on the input. Alias data may identify a second field that is associated with the first field. Using the alias data, a modified query can be generated based on the query and the second field. The modified query can be executed to generate query results. The query results can be displayed via a visualization of the user interface based on the first field.

Generation of modified queries using a field value for different fields

An instrumentation analysis system processes data streams by executing instructions specified using a data stream language program. The data stream language allows users to specify a search condition using a find block for identifying the set of data streams processed by the data stream language program. The set of identified data streams may change dynamically. The data stream language allows users to group data streams into sets of data streams based on distinct values of one or more metadata attributes associated with the input data streams. The data stream language allows users to specify a threshold block for determining whether data values of input data streams are outside boundaries specified using low/high thresholds. The elements of the set of data streams input to the threshold block can dynamically change. The low/high threshold values can be specified as data streams and can dynamically change.

Dynamically changing input data streams processed by data stream language programs

Described are systems, methods, and techniques for collecting, analyzing, processing, and storing time series data and for evaluating and dynamically estimating a resolution of one or more streams of data points and updating an output resolution. Responsive to receiving a stream of data points, a data resolution can be derived and an output resolution can be set to a first value. When a change to the data resolution is detected, the output resolution can be changed, modifying a frequency at which output data points are generated and/or transmitted. In some instances, a detector can be implemented to trigger an alert responsive to ingested data points corresponding with triggering parameters. An output resolution for the detector can be dynamically modified based on dynamically detecting a change to the data resolution of the stream of data.

Dynamic resolution estimation for a detector

Dynamic resolution estimation in metric time series data

Automatic generation of queries using non-textual input

An analysis system receives data streams generated by instances of instrumented software executing on external systems. The analysis system evaluates an expression using data values of the data streams over a plurality of time intervals. For example, the analysis system may aggregate data values of data streams for each time interval. The analysis system determines whether or not a data stream is considered for a time interval based on when the data value arrives during the time interval. The analysis system determines a maximum expected delay value for each data stream being processed. The analysis system evaluates the expression using data values that arrive before their maximum expected delay values. The analysis system also determines a failure threshold value for a data stream. If a data value of a data stream fails to arrive before the failure threshold value, the analysis system marks the data stream as dead.

Real-time processing of data streams received from instrumented software

Systems and methods are described for generation of queries for execution by a separate system. In order establish a connection with the separate system, credentials can be obtained. For example, the credentials may be based on a user identifier and/or a login identifier. Indices can be identified that correspond to the credentials and a query can be identified that includes a selection of at least one of the indices. For example, the query may identify a set of log data ingested and indexed by the separate system. A request that includes the query, the credentials, and a connection identifier can be communicated to the separate system. In response to the request, a set of data can be received from the separate system. The set of data can be provided to a computing device. For example, the set of data can be provided to a computing device providing the query.

Generation of queries for execution at a separate system

A system processes data stream language expressions that combine result data streams from multiple data stream language sub-expressions. The system determines a set of fixed dimensions based on static analysis of the data stream language sub-expression. The system determines a union set representing a union of the sets of fixed dimensions. The system determines at execution time of the data stream language expression, a plurality of sets of data streams. Each set of data stream corresponds to a data stream language sub-expression from the plurality of data stream language expressions. The system correlates data streams across the plurality of sets of data streams based on the union set. The system determines result data streams for the data stream language expression by combining data values of correlated data streams.

Growing community of inventors

Palo Alto, CA, United States of America

Rajesh Raman