Efficient execution of group-sparsified neural networks

Abstract

Creating neural network (NN) code may include for each row in a kernel matrix, finding the first non-zero element; and creating a loop including multiply and add instructions. On each iteration of the loop, the multiply and add instructions may be executed, and the position of the kernel matrix operand operated on by each multiply and add may be correlated to the loop iteration number. Instructions may be issued or created to be executed in the loop. A method may execute a NN by executing a loop including a series of multiply and add instructions to multiply a kernel matrix A by an input, such that on each iteration of the loop the series of multiply and add instructions are executed; and the position of the matrix A operand operated on by each multiply and add instruction in the series is correlated to the iteration number of the loop.