Splunk Inc.

Dragoljub Profirovic

Min Zhang

Zhimin Liang

Sanjeev Raghavendra Kulkarni

Poornima Devaraj

Boyang Peng

Systems and methods are described for implementation by a streaming data processing system of a processing pipeline that obtains data items from one or more data sources. The one or more data sources may include multiple partitions of a topic or multiple topics. The one or more data sources are defined by a data pattern, a group of tenants, or a process. The implementation of the processing pipeline can include identifying the one or more data sources to obtain data items. The processing pipeline can include a plurality of processing tasks to obtain the data items. The streaming data processing system can utilize a workload coordinator to dynamically assign particular data sources to particular processing tasks. The implementation of the processing pipeline can further include executing the plurality of processing tasks to cause processing tasks to obtain data items from assigned data sources and process the data items.

Workload coordinator for providing state credentials to processing tasks of a data processing pipeline

Systems and methods are described for tuning parameter values of a processing pipeline in a streaming data processing system. In order to determine an optimal set of parameter values for a particular processing pipeline, a processing pipeline can be implemented with different sets of parameter values. A performance metric can be measured for each implementation to measure the performance of the processing pipeline with regards to a particular set of parameter values. The performance metrics for each implementation can be compared in order to determine optimal performance metrics. The processing pipeline can be implemented based on an optimal set of parameter values that correspond to the optimal performance metrics.

Growing community of inventors

Coquitlam, Canada

Dragoljub Profirovic