International Business Machines Corporation

Samsung Electronics Co., Ltd.

Stanislaw Andrzej Wozniak

Angeliki Pantazi

Evangelos Stavros Eleftheriou

Thomas Bohnstingl

Jean Fompeyrine

Stefan Abel

Tomas Tuma

Giorgia Dellaferrera

Severin Sidler

Ljubica Cimesa

A neural apparatus for a neural network system may be configured to receive one or more input signals during a decode time period, decode the one or more input signals during the decode time period, resulting in a decoded signal, and upon termination of the decode time period, process the decoded signal using internal neuron dynamics. The processed signal may be used to encode and emit one or more output signals in a subsequent decode time period to another neural apparatus of the neural network system.

Neural apparatus for a neural network system

Training a neural network that comprises nodes and weighted connections between selected ones of the nodes is described herein. A function of a desired activity and a current activity during training results in a feedback signal used for adjusting weight values of the connections. For a weight value update cycle the process determines an importance value for various nodes based on current weight values of the connections and determines an adjustment of the feedback signal specific for each weight value of the connections by a combination of a gradient value derived from the feedback signal for a connection and the determined corresponding element of the adjustment matrix. The updates are applied to the connections during update cycles.

Weight-based local modulation of weight update in neural networks

The present disclosure relates to a neural network system comprising: a controller including a processing unit configured to execute a spiking neural network, and an interface connecting the controller to an external memory. The controller is configured for executing the spiking neural network, the executing comprising generating read instructions and/or write instructions. The interface is configured for: generating read weighting vectors according to the read instructions, coupling read signals, representing the read weighting vectors, into input lines of the memory, thereby retrieving data from the memory, generating write weighting vectors according to the write instructions, coupling write signals, representing the write weighting vectors, into output lines of the memory, thereby writing data into the memory.

Memory-augmented spiking neural network system

The present disclosure relates to a neuromorphic neuron apparatus comprising an output generation block and at least one adaptation block. The apparatus has a current adaptation state variable corresponding to previously generated one or more signals. The output generation block is configured to use an activation function for generating a current output value based on the current adaptation state variable. The adaptation block is configured to repeatedly: compute an adaptation value of its current adaptation state variable using the current output value and a correction function; use the adaption value to update the current adaptation state variable to obtain an updated adaptation state variable, the updated adaptation state variable becoming the current adaptation state variable; receive a current signal; and cause the output generation block to generate a current output value based on the current adaptation state variable and input value that obtained from the received signal.

Adaptive neuromorphic neuron apparatus for artificial neural networks

In a hardware-implemented approach for operating a neural network system, a neural network system is provided comprising a controller, a memory, and an interface connecting the controller to the memory, where the controller comprises a processing unit configured to execute a neural network and the memory comprises a neuromorphic memory device with a crossbar array structure that includes input lines and output lines interconnected at junctions via electronic devices. The electronic devices of the neuromorphic memory device are programmed to incrementally change states by coupling write signals into the input lines based on: write instructions received from the controller and write vectors generated by the interface. Data is retrieved from the neuromorphic memory device, according to a multiply-accumulate operation, by coupling read signals into one or more of the input lines of the neuromorphic memory device based on: read instructions from the controller and read vectors generated by the interface.

Crossbar arrays for computations in memory-augmented neural networks

A neuromorphic neuron apparatus includes an accumulation block and an output generation block. The apparatus has a current state variable corresponding to previously received one or more signals. The output generation block is configured to use an activation function for generating a current output value based on the current state variable. The accumulation block is configured to repeatedly: compute an adjustment of the current state variable using the current output value and a correction function indicative of a decay behaviour of a time constant of the apparatus; receive a current signal; update the current state variable using the computed adjustment and the received signal, the updated state variable becoming the current state variable; and cause the output generation block to generate a current output value based on the current state variable.

Neuromorphic neuron apparatus for artificial neural networks

The present disclosure relates to a storage device comprising a memory element. The memory element may comprise a changeable physical quantity for storing information. The physical quantity may be in a drifted state. The memory element may be configured for setting the physical quantity to an initial state. Furthermore, the memory element may comprise a drift of the physical quantity from the initial state to the drifted state. The initial state of the physical quantity may be computable by means of an initialization function. The initialization function may be dependent on a target state of the physical quantity and the target state of the physical quantity may be approximately equal to the drifted state of the physical quantity.

Method for designing an initialization function for programming a memory element

Neuromorphic processing apparatus is provided. The present invention may include a spiking neural network comprising a set of input spiking neurons each connected to each of a set of output spiking neurons via a respective synapse for storing a synaptic weight which is adjusted for that synapse in dependence on network operation in a learning mode of the apparatus, and each synapse is operable to provide a post-synaptic signal, dependent on its synaptic weight, to its respective output neuron. The present invention may further include a pre-processor unit adapted to process input data, defining a pattern of data points, to produce a first set of input spike signals which encode values representing respective data points, and a second set of input spike signals which encode values complementary to respective said values representing data points, and to supply the input spike signals to respective predetermined input neurons of the network.

Neuromorphic processing apparatus

Embodiment of the invention are directed to transmitting signals between neurons of a hardware-implemented, spiking neural network (or SNN). The network includes neuronal connections, each including a synaptic unit connecting a pre-synaptic neuron to a post-synaptic neuron. Spikes received from the pre-synaptic neuron of said each neuronal connection are first modulated, in frequency, based on a synaptic weight stored on said each synaptic unit, to generate post-synaptic spikes, such that a first number of spikes received from the pre-synaptic neuron are translated into a second number of post-synaptic spikes. At least some of the spikes received from the pre-synaptic neuron may, each, be translated into a train of two or more post-synaptic spikes. The post-synaptic spikes generated are subsequently transmitted to the post-synaptic neuron of said each neuronal connection. The novel approach makes it possible to obtain a higher dynamic range in the synapse output.

Spiking synaptic elements for spiking neural networks

A neuromorphic architecture for a spiking neural network comprising a plurality of spiking neurons, each with a plurality of synapses and corresponding synaptic weights, the architecture further comprising a synaptic competition mechanism in connection with a spike-based learning mechanism based on spikes perceived behind a synapse, in which architecture synapses of different neurons connected to the same input compete for that input and based on the result of that competition, each neuron of the neural network develops an individual perception of the presented input spikes, the perception used by the learning mechanism to adjust the synaptic weights.

Neuromorphic architecture for feature learning using a spiking neural network

This invention relates to an apparatus, system, and method for computing with neuromorphic circuit architectures that have neurons with interconnected internal state information. The interconnected internal state information allows the neurons to enable or strengthen the input to other neurons. Furthermore, neuron internal state information provides insights on the characteristics of the input data that can be used to enhance the performance of the neuromorphic system. The neuromorphic system can be implemented with an artificial phase-change-based neurons.

Neuromorphic architecture with multiple coupled neurons using internal state neuron information

Embodiments relate to a neuromorphic architecture for unsupervised feature learning using memristive synapses realized using phase-change devices. A spiking neural network architecture for unsupervised pattern learning and a spike-based learning algorithm compatible with phase-change synapses is described, and a feature-learning algorithm capable of performing a sequence of set operations on input patterns is provided. A learning rule for the extraction of certain features of the input that is compatible with spiking neurons and synapses with spike-based plasticity is also provided. The system enables enhanced pattern- and feature-extraction capabilities in neuromorphic systems.

Growing community of inventors

Adliswil, Switzerland

Stanislaw Andrzej Wozniak