Message passing interface tuning using collective operation modeling

Abstract

Illustrative embodiments of methods, machine-readable media, and parallel computing systems that automatically tune a communication protocol are disclosed. In some embodiments, a method may include selecting a collective operation to be implemented by a Message Passing Interface (MPI) of a parallel computing system, wherein the MPI is configured to implement the collective operation using at least one of a plurality of algorithms; modeling each of the plurality of algorithms as a corresponding plurality of point-to-point operations; determining a transfer rate of each of the plurality of algorithms using its corresponding plurality of point-to-point operations; selecting one or more algorithms from among the plurality of algorithms, the one or more selected algorithms having higher transfer rates than one or more non-selected algorithms; and testing performance of the one or more selected algorithms with a plurality of configuration parameters to determine at least one optimized algorithm for implementing the collective operation.