International Business Machines Corporation

Nandakumar Sasidharan Rajalekshmi

Evangelos Stavros Eleftheriou

Abu Sebastian

Manuel Le Gallo-Bourdeau

Irem Boybat Kara

Christophe Piveteau

Methods and apparatus are provided for training an artificial neural network, having a succession of neuron layers with interposed synaptic layers each storing a respective set of weights {w} for weighting signals propagated between its adjacent neuron layers, via an iterative cycle of signal propagation and weight-update calculation operations. Such a method includes, for at least one of the synaptic layers, providing a plurality Pof arrays of memristive devices, each array storing the set of weights of that synaptic layer Sin respective memristive devices, and, in a signal propagation operation, supplying respective subsets of the signals to be weighted by the synaptic layer Sin parallel to the Parrays. The method also includes, in a weight-update calculation operation, calculating updates to respective weights stored in each of the Parrays in dependence on signals propagated by the neuron layers.

Training of artificial neural networks

Methods and apparatus are provided for training an artificial neural network having a succession of layers of neurons interposed with layers of synapses. A set of crossbar arrays of memristive devices, connected between row and column lines, implements the layers of synapses. Each memristive device stores a weight for a synapse interconnecting a respective pair of neurons in successive neuron layers. The training method includes performing forward propagation, backpropagation and weight-update operations of an iterative training scheme by applying input signals, associated with respective neurons, to row or column lines of the set of arrays to obtain output signals on the other of the row or column lines, and storing digital signal values corresponding to the input and output signals. The weight-update operation is performed by calculating digital weight-correction values for respective memristive devices, and applying programming signals to those devices to update the stored weights.

A computer-implemented method of mixed-precision deep learning with multi-memristive synapses may be provided. The method comprises representing, each synapse of an artificial neural network by a combination of a plurality of memristive devices, wherein each of the plurality of memristive devices of each of the synapses contributes to an overall synaptic weight with a related device significance, accumulating a weight gradient &#x394;W for each synapse in a high-precision variable, and performing a weight update to one of the synapses using an arbitration scheme for selecting a respective memristive device, according to which a threshold value related to the high-precision variable for performing the weight update is set according to the device significance of the respective memristive device selected by the arbitration schema.

Growing community of inventors

Thalwil, Switzerland

Nandakumar Sasidharan Rajalekshmi