Accelerated neural network training using a pipelined resistive processing unit architecture

Abstract

A neural network system comprises a plurality of neurons, comprising a layer of input neurons, one or more layers of hidden neurons, and a layer of output neurons. The system further comprises a plurality of arrays of weights, each array of weights being configured to receive a plurality of discrete data points from a first layer of neurons and to produce a corresponding discrete data point to a second layer of neurons during a feed forward operation, each array of weights comprising a plurality of resistive processing units (RPU) having respective settable resistances. The system includes a neuron control system configured to control an operation mode of each of the plurality of neurons, wherein the operation mode comprises: a feed forward mode, a back propagation mode, and a weight update mode.