Multivalue reductions using serial initial reductions in multiple register spaces and parallel subsequent reductions in a single register space

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a multivalue reduction using a parallel processing device. One of the methods includes performing a parallel M-value reduction by parallel processing units of a parallel processing device. A plurality of initial reductions are performed in serial, each initial reduction operating on data in a different respective register space of at least M register spaces. Data is moved from the M register spaces so that all results from the plurality of initial reductions are in a same first register space. One or more subsequent reductions are performed in parallel to compute M final values, each subsequent reduction operating only on data in the first register space.