Method, apparatus and computer program to carry out a training procedure in a convolutional neural network

Abstract

A computer-implemented method in a computing network of a number of processing nodes 1 to X, in the computing network neurons of a Convolutional Neural Network (CNN) are divided between the number of nodes. The method including allocating a mini-batch of input data from among mini-batches of input data to a node of the nodes; splitting the mini-batch into a number of mini-batch sections X corresponding and equal to the number of nodes; at the node retaining a mini-batch section which has a same number as the node and sending other mini-batch sections of the split mini-batch sections to corresponding other nodes according to a number of the split mini-batch sections; collating at the node the split mini-batch sections at the node into a single matrix and multiplying the collated matrix by the neurons to provide output data sections having one section of output data per each mini-batch. At the node sending the output data sections corresponding to the other corresponding nodes to the corresponding nodes and combining the output data sections in the node so that the node has output data for entire of the split mini-batch sections.