Neural network inference circuit performing matrix multiplication

Abstract

Some embodiments provide a neural network inference circuit (NNIC) for executing a network having multiple layers. The NNIC includes multiple circuit sets. Each circuit set includes a dot product circuit to compute dot products between weight values and activation values for at least a subset of a first set of the layers, a math function circuit to compute values based on computations using activation values for at least a subset of a second set of the layers, and a post-processing circuit to receive (i) values output by the dot product circuit and (ii) values output by the math function circuit and to perform post-processing operations on the received values. The NNIC includes a set of accumulation circuits. Each accumulation circuit is to accumulate outputs of math function circuits for layers of the second set of layers that perform matrix multiplication of sets of activation values output by previous layers.