Training neural network components

Abstract

A machine learning model may be configured for training using an associated learning technique. A model configured for end-to-end backpropagation may adapted for associated learning by introducing functions for projecting hidden vectors and labels to a shared representation space and for reconstructing labels from representation vectors. An associated learning loss may be calculated at each layer, with the resulting gradients backpropagated locally through that layer rather than all layers. A reconstruction loss may be calculated using each layer's output including the predicted label. Training by associated learning may be parallelized (e.g., layer by layer) to yield efficiency gains. In addition, associated learning training may be more robust to training label errors. The resulting model may be used to, for example, predict data sequences in an autoregressive manner in which subsequent portions of the output data sequence are predicted in part based on previous predicted portions of the output data sequence.