Splunk Inc.

Cisco Technology, Inc.

Zhenghui Xie

Anish Shrigondekar

Ruochen Zhang

Bhavin Thaker

Shalabh Goyal

Igor Stojanovski

Tianyi Gou

Sai Krishna Sajja

Alexandros Batsakis

Ashish Mathew

Tameem Anwar

Mehul Goyal

Sophia Rui Zhu

Douglas Rapp

Eric Timothy Woo

Kartheek Babu Kolla

Kai-Sern Lim

Srinivas Bobba

Srinivas Chowdhary Bobba

A data intake and query system can manage the search of large amounts of data using one or more processing nodes. When a new processing node is added or becomes available, the node coordinator can reassign duties from one or more processing nodes to the new processing node. The node coordinator can initially assign the new processing node one or more groups of data for backup purposes. At a later time, the node coordinator can reassign the new processing node to the one or more groups of data for searching purposes.

Reassignment of processing nodes

A data intake and query system can manage the search of large amounts of data using one or more processing nodes. The data intake and query system can identify a first group of processing nodes and cause a first processing node of the group to download and search a particular data group based on a first node map. The data intake and query system may identify a second group of processing nodes that includes the first group of processing nodes and a second processing node. The data intake and query system can transmit commands to cancel one or more data group downloads at the first processing node and can reassign the corresponding data groups for download to the second processing node based on a second node map.

Reassigning data groups based on activation of a processing node

As an indexer indexes and groups events, it can generate data slices that include events. Based on a slice rollover policy, the indexer can add a particular slice to an aggregate slice. Based on an aggregate slice backup policy, the indexer can store a copy of the aggregate slice to a shared storage system. The aggregate slice can be used for restore purposes in the event the indexer fails or becomes unresponsive.

Indexing data at a data intake and query system based on a node capacity threshold

A data intake and query system can manage the search of large amounts of data using one or more processing nodes. The data intake and query system can identify a group of processing nodes and assign a first processing node of the group to download and search a particular data group based on a first node map. The data intake and query system may identify an action associated with the first processing node. The data intake and query system can cause a particular processing node of the group to download the particular data group based on a second node map and transmit an authorization to perform the action to the first processing node.

Deactivating a processing node based on assignment of a data group assigned to the processing node

Reassigning a processing node from downloading to searching a data group

An actionable event collector in a server cluster receives information specifying an actionable event instance regarding an actionable event occurrence in the server cluster. The actionable event collector transmits a representation of the actionable event instance to an actionable event queue builder. The actionable event queue builder inserts the representation as an entry into an actionable event queue. The event action dispatcher processes the entry from the actionable event queue, wherein processing the entry comprises determining a responsive action for the entry and causing performance of the responsive action.

Actionable event responder architecture

A data intake and query system can ingest and index large amounts of data using one or more ingestors and indexers. The ingestors can ingest incoming data and use it to generate events. The ingestor can group the events and prepare them for communication to a message bus. The ingestor can determine a size of the group of events. If the size of the group of events satisfies a message size threshold, the ingestor can store the group of events to a data store, obtain a reference to the group of events, and communicate the reference to the group of events to a message queue. An indexer can obtained the reference from the message queue and use the reference to obtain the group of events from the data store.

Using a data store and message queue to ingest data for a data intake and query system

Reassigning data groups from backup to searching for a processing node

An actionable event responder performs actions including a server cluster determining an actionable event occurrence from a member of a server cluster, determining an event-type code matching the actionable event occurrence, retrieving an event-type response map entry matching the event-type code, and determining, in response to determining that the event-type response map entry matches the event-type code, a response action signifier in the event-type response map entry. The response action signifier indicates a response action performable by the server cluster. The server cluster further detects whether a preauthorization value is set in a dispatch field of the event-type response map entry, and generates an invocation message to a resolution handler based on the preauthorization value being set in the dispatch field of the event-type response map entry. The resolution handler performs the response action to the actionable event occurrence.

Actionable event responder

An operational event processor receives information describing operational events arising in relation to a server cluster of a data intake and query system. Responsive actions are identified for the events that are actionable, and those responsive actions are automatically dispatched. Dispatching may include automatically causing the performance of the actions based on remembered preauthorizations, or automatically causing the performance of the actions based on obtaining real time user authorizations. The operational event processor may employ mechanisms to provide extensibility and a high degree of interoperability with other system components.

Growing community of inventors

Cupertino, CA, United States of America

Zhenghui Xie