Leverage offload programming model for local checkpoints

Abstract

Methods, apparatus, and systems for leveraging an offload programming model for local checkpoints. Compute entities in a computing environment are implemented as one or more sources and a larger number of sinks. A job dispatcher dispatches jobs comprising executable code to the source(s), and the execution of the job code is managed by the source(s). Code sections in the job code designated for offload are offloaded to the sinks by creating offload context information. In conjunction with each offload, an offload object is generated and written to storage. The offloaded code sections are executed by the sinks, which return result data to the source, e.g., via a direct write to a memory buffer specified in the offload context information. The health of the sinks is monitored to detect failures, and upon a failure the source retrieves the offload object corresponding to the code section offloaded to the failed sink, regenerates the offload context information for the code section and sends this to another sink for execution.