Scheduler for tall-gathering algorithms that include control flow statements

Abstract

A scheduler is provided for tall algorithms that include control flow statements that are dependent on tall computations. Thunks may be defined for the tall algorithms and the boundaries or limits of the thunks may be determined by places where tall variables are required to be gathered. The scheduler executes the tall algorithms by calling individually the thunks from the tall algorithms and pausing when a gather operation is required. The scheduler collectively gathers the tall variables from different tall algorithms by using an optimizer to combine I/O operations for the tall algorithms. The scheduler may put back the gathered variables into the context of each tall algorithm, and the scheduler may resume with the next thunk for each tall algorithm. This process may be repeated until all tall algorithms are finished.