Splunk Inc.

Splunkinc.

Mitchell Neuman Blank

Anish Shrigondekar

Bharath Kishore Reddy Aleti

Rama Gopalan

Hongxun Liu

Alexandros Batsakis

Alexandras Batsakis

A computer implemented system is described for assigning executable jobs to pipeline sets, whereby the jobs may be network based computer jobs. The assigning includes generating a weight for each pipeline set of multiple pipeline sets to obtain multiple weights. Generating a weight includes obtaining duty cycle metrics for pipeline software threads in the pipeline set. The duty cycle metrics include a measure of an amount of time that a corresponding pipeline thread is executing and actively processing data. Generating the weight further includes determining the weight for the pipeline set based at least in part on the duty cycle metrics. The method further includes assigning a job request to a target pipeline set selected from the pipeline sets according to a weighted random algorithm, wherein the weighted random algorithm uses the weights.

Duty cycle estimation for job assignment

Resource management includes storing, for multiple workload pools of a data intake and query system, a workload pool hierarchy arranged in multiple workload pool layers. After storing a processing request is assigned a selected subset of workload pools in a second layer of the workload pool hierarchy based on a type of processing request. The processing request is then assigned to an individual workload pool in the selected subset to obtain a selected workload pool. Execution of the processing request is initiated on the selected workload pool.

Workload pool hierarchy for a search and indexing system

Dynamic reassignment of search processes into workload pools includes receiving a search query to search at least one data store, assigning the search query to a first workload pool, and executing the search query using a first hardware resource in the first workload pool, the first hardware resource corresponding to a first portion of a hardware device. Dynamic reassignment further includes receiving, while executing the search query, an update command to move the search query to a second workload pool, moving, while executing the search query, the search query to the second workload pool; and continuing execution of the search query using a second hardware resource in the second workload pool. The second hardware resource corresponds to a second portion of the hardware device.

Dynamic reassignment in a search and indexing system

Pipeline set selection based on duty cycle estimation

Achieving search and ingest isolation via resource management in a search and indexing system includes receiving a search query associated with at least one data store, assigning, in response to the search query being associated with the at least one data store, the search query to a first workload pool in a set of query workload pools, and processing the search query using a first hardware resource in the first workload pool. Achieving search and ingest isolation further includes receiving an ingest request comprising data associated with the at least one data store. The ingest request is assigned to a second workload pool in a set of ingest workload pools. The set of query workload pools and the set of ingest workload pools are disjoint. Achieving search and ingest isolation further includes processing the ingest request using a second hardware resource in the second workload pool.

Achieve search and ingest isolation via resource management in a search and indexing system

Dynamic reassignment of search processes into workload pools in a search and indexing system

Pipeline set selection based on duty cycle estimation of pipeline threads

Growing community of inventors

London, United Kingdom

Mitchell Neuman Blank