Sharded polling system

Abstract

A shard polling system fairly distributes stored items from producers to consumer processes and includes polling threads that poll for items from respective portions of a storage source, place the items in respective queues, and increment a global permit counter, restricted to a configurable maximum, that tracks the quantity of messages across the respective queues. The polling threads are restricted by respective shard permit counters that limit a quantity of items that may be moved from a storage source to a respective queue. A global thread applies a fairness technique, such as round robin, to fairly obtain the items from each of the respective queues and places the items in a global queue, decrementing the global permit counter, and incrementing the respective shard permit counter for each item obtained. Consumer threads poll the global queue (e.g., a blocking output queue) for the messages.