Assigning and scheduling threads for multiple prioritized queues

Abstract

An operating system provides a pool of worker threads servicing multiple queues of requests at different priority levels. A concurrency controller limits the number of currently executing threads. The system tracks the number of currently executing threads above each priority level, and preempts operations of lower priority worker threads in favor of higher priority worker threads. A system can have multiple pools of worker threads, with each pool having its own priority queues and concurrency controller. A thread also can change its priority mid-operation. If a thread becomes lower priority and is currently active, then steps are taken to ensure priority inversion does not occur. In particular, the current thread for the now lower priority item can be preempted by a thread for a higher priority item and the preempted item is placed in the lower priority queue.