Splunk Inc.

Ningxuan He

Alexandros Batsakis

Balaji Rao

Nitilaksha Halakatti

Manuel Martinez

Sourav Pal

Arindam Bhattacharjee

Jindrich Dinga

Kieran Nicholas Cairney

Jianming Zhang

Steve Yu Zhang

Prem Kumar Jayaraj

A computing device can receive a query that identifies a set of data to be processed and determine that a portion of the set of data resides in an external data system. The query system can request data identifiers associated with data objects of the set of data from the external data system and communicate the data identifiers to a data queue. The computing device can instruct one or more search nodes to retrieve the identifiers from the data queue. The search nodes can use the data identifiers to retrieve data objects from the external data system and process the data objects according to instructions received from the computing device. The search nodes can provide results of the processing to the computing device.

Retrieving data identifiers from queue for search of external data system

Systems and methods are described for establishing and managing components of a distributed computing framework implemented in a data intake and query system. The distributed computing framework may include a master and a plurality of worker nodes. The master may selectively operate on a search head captain that is chosen from the search heads of the data intake and query system. The search head captain may distribute configuration information for the master and the distributed computing framework to the other search heads, which in turn, may distribute that configuration information to indexers of the data intake and query system. Worker nodes may be selectively activated for operation on the indexers based on the configuration information, and the worker nodes may additionally use the configuration information to contact the master and join the distributed computing framework. This approach may provide numerous benefits, including improved security, flexibility in the selection of worker nodes, and redundancy for failures of physical components of the data intake and query system.

Management of distributed computing framework components

Management of distributed computing framework components in a data fabric service system

Growing community of inventors

Sunnyvale, CA, United States of America

Ningxuan He