Shuffle-based request buffer for managing large request volumes

Abstract

Approaches are disclosed for managing aspects of content delivery in a multi-tenant environment. A request buffer can be used to remove correlations between requests and randomly shuffle requests without storing all the requests concurrently. A shuffle sharding algorithm can be used to randomly allocate a subset of resources to different users in order to ensure less than a maximum risk of one user impacting the use of all resources allocated to other users. In some embodiments, separate fleets of resources can be maintained for manifests and video segments to allow for more accurate scaling and customization. Multiple manifests can also be associated with a single endpoint to allow multiple media players to obtain similar content segments from the single endpoint.