Partitioning for delayed queues in a distributed network

Abstract

A timer service receives timer creation requests from clients and fires those timers upon their expiration by returning to the requesting clients payloads provided with the timer creation requests. A timer creation request includes a client identifier, a timer expiration time, and a payload. The timer service queues timer creation requests, a creation worker adds timer records to a timer store in response to requests and manages partitions of the timer store, and sweeper workers, one per partition, sweep timers into a second queue. A firing worker reads timer index records from the second queue, determines the timers referenced in those timer index records and executes the payloads of those timers. The timer store can be tiered, with partitions different storage priorities based on the timer expirations of the timers in the partitions.