Neural network internal data fast access memory buffer

Abstract

Systems, apparatuses, and methods for optimizing neural network training with a first-in, last-out (FILO) buffer are disclosed. A processor executes a training run of a neural network implementation by performing multiple passes and adjusting weights of the neural network layers on each pass. Each training phase includes a forward pass and a backward pass. During the forward pass, each layer, in order from first layer to last layer, stores its weights in the FILO buffer. An error is calculated for the neural network at the end of the forward pass. Then, during the backward pass, each layer, in order from last layer to first layer, retrieves the corresponding weights from the FILO buffer. Gradients are calculated based on the error so as to update the weights of the layer for the next pass through the neural network.