International Business Machines Corporation

Richard Michael Shok

Philip James Sanders

David Lee Darrington

Matthew W Markland

Thomas M Gooding

Efficient application checkpointing uses checkpointing characteristics of a job to determine how to schedule jobs for execution on a multi-node computer system. A checkpoint profile in the job description includes information on the expected frequency and duration of a check point cycle for the application. The checkpoint profile may be based on a user/administrator input as well as historical information. The job scheduler will attempt to group applications (jobs) that have the same checkpoint profile, on the same nodes or group of nodes. Additionally, the job scheduler may control when new jobs start based on when the next checkpoint cycle(s) are expected. The checkpoint monitor will monitor the checkpoint cycles, updating the checkpoint profiles of running jobs. The checkpoint monitor will also keep track of an overall system checkpoint profile to determine the available checkpointing capacity before scheduling jobs on the cluster.

Scheduling work in a multi-node computer system based on checkpoint characteristics

A hybrid node of a High Performance Computing (HPC) cluster uses accelerator nodes for checkpointing to increase overall efficiency of the multi-node computing system. The host node or processor node reads/writes checkpoint data to the accelerators. After offloading the checkpoint data to the accelerators, the host processor can continue processing while the accelerators communicate the checkpoint data with the host or wait for the next checkpoint. The accelerators may also perform dynamic compression and decompression of the checkpoint data to reduce the checkpoint size and reduce network loading. The accelerators may also communicate with other node accelerators to compare checkpoint data to reduce the amount of checkpoint data stored to the host.

Using accelerators in a hybrid architecture for system checkpointing

A debugger debugs processes that execute shared instructions so a breakpoint set for one process will not cause a breakpoint to occur in the other processes. A breakpoint is set by recording the original instruction at the desired location and writing a trap instruction to the shared instructions at that location. When a process encounters the breakpoint, the process passes control to the debugger for breakpoint processing if the breakpoint was set at that location for that process. If the trap was not set at that location for that process, the cacheline containing the trap is copied to a small scratchpad memory, and the virtual memory mappings are changed to translate the virtual address of the cacheline to the scratchpad. The original instruction is then written to replace the trap instruction in the scratchpad, so the process can execute the instructions in the scratchpad thereby avoiding the trap instruction.

Handling debugger breakpoints in a shared instruction system

A method, apparatus, and program product checkpoint an application in a parallel computing system of the type that includes a plurality of hybrid nodes. Each hybrid node includes a host element and a plurality of accelerator elements. Each host element may include at least one multithreaded processor, and each accelerator element may include at least one multi-element processor. In a first hybrid node from among the plurality of hybrid nodes, checkpointing the application includes executing at least a portion of the application in the host element and at least one accelerator element and, in response to receiving a command to checkpoint the application, checkpointing the host element separately from the at least one accelerator element.

Checkpointing a hybrid architecture computing system

A method, apparatus, and program product checkpoint an application in a parallel computing system of the type that includes a plurality of hybrid nodes. Each hybrid node includes a host element and a plurality of accelerator elements. Each host element may include at least one multithreaded processor, and each accelerator element may include at least one multi-element processor. In a first hybrid node from among the plurality of hybrid nodes, checkpointing the application includes executing at least a portion of the application in the host element, configuring and executing at least one computation kernel in at least one accelerator element, and, in response to receiving a command to checkpoint the application, checkpointing the host element separately from the at least one accelerator element upon which the at least one computation kernel is executing.

Growing community of inventors

Rochester, MN, United States of America

Richard Michael Shok