Automatic recovery of reservoir simulation runs from processing system failures

Abstract

Reservoir simulation is performed for giant reservoir models in a parallel computing platform composed of a number of processor nodes. Automatic precautionary checkpoints are made at regular time intervals when computational time exceeds a preset value. The simulator receives and reacts to signals from a real time monitoring interface tool which monitors the health of the system. Checkpoints are also made done if a system problem which may cause a simulation job to fail is projected. The simulation job is subsequently restarted to continue simulation from the last checkpoint. The monitoring and automatic recovery are done automatically without need for user intervention.