Commissariat a L'energie Atomique Et Aux Energies Alternatives

Centre National De La Recherche Scientifique

National Institute of Health and Medical Research

Olivier Bichler

Vincent Lorrain

Johannes Christian Thiele

Christian Gamrat

Elisa Vianello

Antoine Dupret

Marc Duranton

Manan Suri

Damien Querlioz

David Briand

Jean-Marc Philippe

Alexandre Carbon

Barbara De Salvo

Blaise Yvert

Barbara Desalvo

Thilo Werner

An electronic calculator comprises a plurality of electronic calculation blocks, each of which is configured to implement one or more respective processing layers of an artificial neural network. The calculation blocks are of at least two different types among: a first type with fixed topology, fixed operation, and fixed parameters, a second type with fixed topology, fixed operation, and modifiable parameters, and a third type with modifiable topology, modifiable operation, and modifiable parameters. For each processing layer implemented by the respective calculation block, the topology is a connection topology for each artificial neuron, the operation is a type of processing to be performed for each artificial neuron, and the parameters include values determined via training of the neural network.

Electronic calculator for the implementation of an artificial neural network, with calculation blocks of several types

A new implementation is provided for an error back-propagation algorithm that is suited to the hardware constraints of a device implementing a spiking neural network. The invention notably uses binary or ternary encoding of the errors calculated in the back-propagation phase to adapt its implementation to the constraints of the network, and thus to avoid having to use floating-point number multiplication operators. More generally, the invention proposes a global adaptation of the back-propagation algorithm to the specific constraints of a spiking neural network. In particular, the invention makes it possible to use the same propagation infrastructure to propagate the data and to back-propagate the errors in the training phase. The invention proposes a generic implementation of a spiking neuron that is suitable for implementing any type of spiking neural network, in particular convolutional networks.

Backpropagation of errors in pulsed form in a pulsed neural network

A computer-implemented method for coding a digital signal intended to be processed by a digital computing system includes the steps of: receiving a sample of the digital signal quantized on a number Nof bits, decomposing the sample into a plurality of binary words of parameterizable bit size N, coding the sample through a plurality of pairs of values, each pair comprising one of the binary words and an address corresponding to the position of the binary word in the sample, transmitting the pairs of values to an integration unit in order to carry out a MAC operation between the sample and a weighting coefficient.

Method and device for binary coding of signals in order to implement digital MAC operations with dynamic precision

A memory circuit for storing parsimonious data and intended to receive an input vector of size Iz, includes an encoder, a memory block comprising a first memory region and a second memory region divided into a number Iz of FIFO memories, each FIFO memory being associated with one component of the input vector, only non-zero data being saved in the FIFO memories, a decoder, the encoder being configured to generate an indicator of non-zero data for each component of the input vector, the memory circuit being configured to write the non-zero data of the input data vector to the respective FIFO memories and to write the indicator of non-zero data to the first memory region, the decoder being configured to read the outputs of the FIFO memories and the associated indicator in the first memory region.

Memory circuit for storing parsimonious data

A modulation device includes at least one memristive device, and a control block, the modulation device having an equivalent conductance y(t) produced by the at least one memristive device and the control block being configured to receive a clock signal and perform a first modification of the equivalent conductance y(t) upon receipt of each clock signal, receive an input voltage pulse and perform a second modification of the equivalent conductance y(t) upon receipt of each input voltage pulse, the first and second modifications being in opposite directions.

Modulation device and method, artificial synapse comprising said modulation device, short term plasticity method in an artificial neural network comprising said artificial synapse

A processor for computing at least one convolution layer of a convolutional neural network is provided, in response to an input event, the convolutional neural network comprising at least one convolution kernel, the convolution kernel containing weight coefficients. The processor comprises at least one convolution module configured to compute the one or more convolution layers, each convolution module comprising a set of elementary processing units for computing the internal value of the convolution-layer neurons that are triggered by the input event, each convolution module being configured to match the weight coefficients of the kernel with certain at least of the elementary processing units of the module in parallel, the number of elementary processing units being independent of the number of neurons of the convolution layer.

Device and method for calculating convolution in a convolutional neural network

A computer based on a spiking neural network, includes at least one maximum pooling layer. In response to an input spike received by a neuron of the maximum pooling layer, the device is configured so as to receive the address of the activated synapse. The device comprises an address comparator configured so as to compare the address of the activated synapse with a set of reference addresses. Each reference address is associated with a hardness value and with a pooling neuron. The device activates a neuron of the maximum pooling layer if the address of the activated synapse is equal to one of the reference addresses and the hardness value associated with this reference address has the highest value from among the hardness values associated with the other reference addresses of the set.

Computer for spiking neural network with maximum aggregation

A device for distributing the convolution coefficients of the least one convolutional kernel of a convolutional neural network is provided, the coefficients being carried by an input bus, to a set of processing units in a processor based on a convolutional-neural-network architecture. The device comprises at least one switching network that is controlled by at least one control unit, the switching network comprising a set of switches that are arranged to apply circular shifts to at least one portion of the input bus. For each convolution kernel, each control unit is configured to dynamically control certain at least of the switches of the switching networks in response to an input event applied to the convolution kernel and at least one parameter representing the maximum size of the convolution kernels.

Device and method for distributing convolutional data of a convolutional neural network

A method for unsupervised learning of a multilevel hierarchical network of artificial neurons wherein each neuron is interconnected with artificial synapses to neurons of a lower hierarchical level and to neurons of an upper hierarchical level. The method includes at a neuron the steps of integrating inference spikes from the interconnected neurons of the lower hierarchical level both in a first and a second accumulators using the same synaptic weights; when the first accumulator reaches a first threshold, generating a learning spike, resetting the first accumulator, triggering synaptic conductance modification in accordance with a spike-timing dependent plasticity rule and delivering the learning spike as an inhibitory signal to other neurons in the same hierarchical level; when the second accumulator reaches a second threshold, generating an inference spike, delivering the generated inference spike to the interconnected neurons of the upper hierarchical level, resetting the second accumulator and possibly delivering the inference spike as an inhibitory signal to other neurons in the same hierarchical level.

STDP-based learning method for a network having dual accumulator neurons

A neuron circuit is capable of producing a weighted sum of digitized input signals and applying an activation function to the weighted sum so as to produce a digitized activation signal as output. The circuit includes at least: one multiplier multiplying each input signal (xto x) with a weight value (wto w), one accumulator accumulating the results of the multiplier so as to produce the weighted sum, and one activation unit executing the activation function. The activation unit comprises at least one shift unit and at least one saturation unit capable of approximating a non-linear activation function. The result of the approximated activation function is obtained by one or more arithmetic shifts applied to the weighted sum.

Optimized neuron circuit, and architecture and method for executing neural networks

A convolutional neural network is provided comprising artificial neurons arranged in layers, each comprising output matrices. An output matrix comprises output neurons and is connected to an input matrix, comprising input neurons, by synapses associated with a convolution matrix comprising weight coefficients associated with the output neurons of an output matrix. Each synapse consists of a set of memristive devices storing a weight coefficient of the convolution matrix. In response to a change of the output value of an input neuron, the neural network dynamically associates each set of memristive devices with an output neuron connected to the input neuron. The neural network comprises accumulator(s) for each output neuron; to accumulate the values of the weight coefficients stored in the sets of memristive devices dynamically associated with the output neuron, the output value of the output neuron being determined from the value accumulated in the accumulator(s).

Convolutional neural network

A programming method for an artificial neuron network having synapses, each including a single resistive random-access memory having first and second electrodes on either side of an active zone, the method including determining a number N of conductance intervals, where N&#x2265;3; for each memory: choosing a conductance interval from amongst the N intervals; a step i) for application of a voltage pulse of a first type between the first and second electrodes, and for reading the conductance value of the memory; if the conductance value does not belong to the previously chosen conductance interval, a sub-step ii) for application of a voltage pulse of a second type between the first and second electrodes, and for reading the conductance value; if the conductance value does not belong to the chosen conductance interval, a step according to which step i) is reiterated, with steps i) and ii) being repeated until the conductance value belongs to the interval.

Method of programming an artificial neuron network

A neural network comprises a plurality of artificial neurons and a plurality of artificial synapses each input neuron being connected to each output neuron by way of an artificial synapse, the network being characterized in that each synapse consists of a first memristive device connected to a first input of an output neuron and of a second memristive device, mounted in opposition to said first device and connected to a second, complemented, input of said output neuron so that said output neuron integrates the difference between the currents originating from the first and second devices.

Network of artificial neurons based on complementary memristive devices

A neuromorphic system comprises a set of at least one input neuron, a set of at least one output neuron and a synaptic network formed from a set of at least one variable-resistance memristive component, said synaptic network connecting at least one input neuron to at least one output neuron, the resistance of the at least one memristive component being adjusted by delivering to the synaptic network write pulses generated by the at least one input neuron, and return pulses generated by the at least one output neuron, the characteristics of the write and return pulses being deduced from the intrinsic characteristics of the at least one memristive component so that the combination of a write pulse and a return pulse in the at least one memristive component results in a modification of its resistance according to a learning rule chosen beforehand.

Neuromorphic system exploiting the intrinsic characteristics of memory cells

An unsupervised learning method is provided implemented in an artificial neural network based on memristive devices. It consists notably in producing an increase in the conductance of a synapse when there is temporal overlap between a pre-synaptic pulse and a post-synaptic pulse and in decreasing its conductance on receipt of a post-synaptic pulse when there is no temporal overlap with a pre-synaptic pulse.

Method for non-supervised learning in an artificial neural network based on memristive nanodevices, and artificial neural network implementing said method

Growing community of inventors

Massy, France

Olivier Bichler