International Business Machines Corporation

Hitachi Global Storage Technologies Netherlands B.v.

Globalfoundries Inc.

Hgst Netherlands, B.v.

Samsung Electronics Co., Ltd.

Skhynix Inc.

Lenovo (singapore) Pte. Ltd.

Evangelos Stavros Eleftheriou

Robert Haas

Roy Daron Cideciyan

Abu Sebastian

Roman Alexander Pletka

Sedat Oelcer

Charalampos Pozidis

Giovanni Cherubini

Ioannis Koltsidas

Angeliki Pantazi

Richard Leo Galbraith

Ajay Dholakia

Thomas Mittelholzer

Lokesh M Gupta

Michael T Benhase

Xiao-Yu Hu

Jens Jelitto

Robert Allan Hutchins

Robert Allen Hutchins

Stephen LaRoux Blinick

Xiao-yu Hu

Matthew J Kalos

Manuel Le Gallo-Bourdeau

Glen Alan Jaquette

Jonathan Darrel Coker

Ilias Iliadis

Mark Alfred Lantz

Malcolm Scott Ware

Travis Roger Oenning

Paul Joseph Seger

Gordon Taylor Davis

Sri Muthuthamby Sri-Jayantha

Jeffrey Haskell Derby

Xiaoyu Hu

Nikolaos Papandreou

Peter Mueller

Tomas Tuma

Theodore Antonakopoulos

Hien Phu Dang

Arun Sharma

Gerd K Binnig

Fredy Daniel Neeser

David James Stanek

Thomas R Albrecht

Hua Ye

Dongming Hwang

Youssef Abdelilah

Nhan Xuan Bui

Karl Allen Nielsen

Keisuke Tanaka

Weldon Mark Hanson

Pier Andrea Francese

Lukas Kull

Pierre R Chevillat

Konstantinos Bekas

Alessandro Curioni

Walter Hirt

Slavisa Sarafijanovic

Hisato Matsuo

Kevin J Ash

Nils Haustein

Andrew Dale Walls

Charles John Camp

Adrian Cuenin Gerhard

Bruce Alexander Wilson

Robert Edward Galbraith

Mario Blaum

Reed Alan Hancock

Yu-Cheng Hsu

Michel Despont

Peter Vettiger

Johannes Georg Bednorz

Laurent A Dellmann

Harald Seipp

Daniel Krebs

Ksenija Lakovic

Daniele Caimi

Robert G Biskeborn

Dharmendra S Modha

Nikolas Ioannou

Joseph W Dain

Dean Hildebrand

Ernest Stewart Gale

Robert Beverley Basham

Thomas P Parnell

Yuan Xing Lee

Joseph S Hyde, Ii

Thomas Helmut Toifl

Lawrence Yiumchee Chiu

Gary A Tressler

Paul Henri Muench

Walter Haeberle

Frank Ray Keyser, Iii

Thomas D Weigold

Alfred Emilio Sanchez

Milos Stanisavljevic

Mark Allan Taylor

John Jeffrey Stephenson

Katherine Tyldesley Blinick

Flavio Alvarenga Bergamaschi

A computer-implemented method for answering a cognitive query from sensor input signals may be provided. The method comprises feeding sensor input signals to an input layer of an artificial neural network comprising a plurality of hidden neuron layers and an output neural layer, determining hidden layer output signals from each of the plurality of hidden neuron layers and output signals from the output neural layer, and generating a set of pseudo-random bit sequences by applying a set of mapping functions using the output signals of the output layer and the hidden layer output signals of one of the hidden neuron layers as input data for one mapping function. Furthermore, the method comprises determining a hyper-vector using the set of pseudo-random bit sequences, and storing the hyper-vector in an associative memory, in which a distance between different hyper-vectors is determinable.

Answering cognitive queries from sensor input signals

A memory controller circuit for mapping data of a convolutional neural network to a physical memory is disclosed. The memory controller circuit comprises a receiving unit to receive a selection parameter value, and a mapping unit to map pixel values of one layer of the convolutional neural network to memory words of the physical memory according to one of a plurality of mapping schemas, wherein the mapping is dependent on the value of the received selection parameter value.

Memory mapping of activations for convolutional neural network executions

Training a neural network that comprises nodes and weighted connections between selected ones of the nodes is described herein. A function of a desired activity and a current activity during training results in a feedback signal used for adjusting weight values of the connections. For a weight value update cycle the process determines an importance value for various nodes based on current weight values of the connections and determines an adjustment of the feedback signal specific for each weight value of the connections by a combination of a gradient value derived from the feedback signal for a connection and the determined corresponding element of the adjustment matrix. The updates are applied to the connections during update cycles.

Weight-based local modulation of weight update in neural networks

In an approach, a process stores a matrix of multibit values for a computation in an analog multiply-accumulate unit including at least one crossbar array of binary analog memory cells connected between respective pairs of word- and bit-lines of the array, where: bits of each multibit value are stored in cells connected along a word-line, and corresponding bits of values in a column of the matrix are stored in cells connected along a bit-line. In each of one or more computation stages for a cryptographic element, the process supplies a set of polynomial coefficients of an element bitwise to respective word-lines of the unit to obtain analog accumulation signals on the respective bit-lines. The process converts the analog signals to digital. The process processes the digital signals obtained from successive bits of the polynomial coefficients in each of the stages to obtain a computation result for the cryptographic element.

In-memory computation in homomorphic encryption systems

The present disclosure relates to a neural network system comprising: a controller including a processing unit configured to execute a spiking neural network, and an interface connecting the controller to an external memory. The controller is configured for executing the spiking neural network, the executing comprising generating read instructions and/or write instructions. The interface is configured for: generating read weighting vectors according to the read instructions, coupling read signals, representing the read weighting vectors, into input lines of the memory, thereby retrieving data from the memory, generating write weighting vectors according to the write instructions, coupling write signals, representing the write weighting vectors, into output lines of the memory, thereby writing data into the memory.

Memory-augmented spiking neural network system

Neuromorphic methods, systems and devices are provided. The embodiment may include a neuromorphic device which may comprise a crossbar array structure and an analog circuit. The crossbar array structure may include N input lines and M output lines interconnected at junctions via N&#xd7;M electronic devices, which, in preferred embodiments, include, each, a memristive device. The input lines may comprise Nfirst input lines and Nsecond input lines. The first input lines may be connected to the M output lines via N&#xd7;M first devices of said electronic devices. Similarly, the second input lines may be connected to the M output lines via N&#xd7;M second devices of said electronic devices. The analog circuit may be configured to program the electronic devices so as for the first devices to store synaptic weights and the second devices to store neuronal states.

Neuromorphic device with crossbar array structure storing both weights and neuronal states of neural networks

In a hardware-implemented approach for operating a neural network system, a neural network system is provided comprising a controller, a memory, and an interface connecting the controller to the memory, where the controller comprises a processing unit configured to execute a neural network and the memory comprises a neuromorphic memory device with a crossbar array structure that includes input lines and output lines interconnected at junctions via electronic devices. The electronic devices of the neuromorphic memory device are programmed to incrementally change states by coupling write signals into the input lines based on: write instructions received from the controller and write vectors generated by the interface. Data is retrieved from the neuromorphic memory device, according to a multiply-accumulate operation, by coupling read signals into one or more of the input lines of the neuromorphic memory device based on: read instructions from the controller and read vectors generated by the interface.

Crossbar arrays for computations in memory-augmented neural networks

Methods and apparatus are provided for training an artificial neural network having a succession of neuron layers with interposed synaptic layers each having a respective set of N-bit fixed-point weights {w} for weighting signals propagated between its adjacent neuron layers, via an iterative cycle of signal propagation and weight-update calculation operations. Such a method includes, for each synaptic layer, storing a plurality p of the least-significant bits of each N-bit weight w in digital memory, and storing the next n-bit portion of each weight w in an analog multiply-accumulate unit comprising an array of digital memory elements. Each digital memory element comprises n binary memory cells for storing respective bits of the n-bit portion of a weight, where n&#x2265;1 and (p+n+m)=N where m&#x2265;0 corresponds to a defined number of most-significant zero bits in weights of the synaptic layer.

Training of artificial neural networks

The present disclosure relates to a storage device comprising a memory element. The memory element may comprise a changeable physical quantity for storing information. The physical quantity may be in a drifted state. The memory element may be configured for setting the physical quantity to an initial state. Furthermore, the memory element may comprise a drift of the physical quantity from the initial state to the drifted state. The initial state of the physical quantity may be computable by means of an initialization function. The initialization function may be dependent on a target state of the physical quantity and the target state of the physical quantity may be approximately equal to the drifted state of the physical quantity.

Method for designing an initialization function for programming a memory element

Methods and apparatus are provided for training an artificial neural network, having a succession of neuron layers with interposed synaptic layers each storing a respective set of weights {w} for weighting signals propagated between its adjacent neuron layers, via an iterative cycle of signal propagation and weight-update calculation operations. Such a method includes, for at least one of the synaptic layers, providing a plurality Pof arrays of memristive devices, each array storing the set of weights of that synaptic layer Sin respective memristive devices, and, in a signal propagation operation, supplying respective subsets of the signals to be weighted by the synaptic layer Sin parallel to the Parrays. The method also includes, in a weight-update calculation operation, calculating updates to respective weights stored in each of the Parrays in dependence on signals propagated by the neuron layers.

Methods and apparatus are provided for training an artificial neural network having a succession of layers of neurons interposed with layers of synapses. A set of crossbar arrays of memristive devices, connected between row and column lines, implements the layers of synapses. Each memristive device stores a weight for a synapse interconnecting a respective pair of neurons in successive neuron layers. The training method includes performing forward propagation, backpropagation and weight-update operations of an iterative training scheme by applying input signals, associated with respective neurons, to row or column lines of the set of arrays to obtain output signals on the other of the row or column lines, and storing digital signal values corresponding to the input and output signals. The weight-update operation is performed by calculating digital weight-correction values for respective memristive devices, and applying programming signals to those devices to update the stored weights.

The present disclosure relates to a method for executing a computation task composed of at least one set of operations where subsets of pipelineable operations of the set of operations are determined in accordance with a pipelining scheme. A single routine may be created for enabling execution of the determined subsets of operations by a hardware accelerator. The routine has, as arguments, a value indicative of input data and values of configuration parameters of the computation task, where a call of the routine causes a scheduling of the subsets of operations on the hardware accelerator in accordance with the values of the configuration parameters. Upon receiving input data of the computation task, the routine may be called to cause the hardware accelerator to perform by the computation task in accordance with the scheduling.

Method for interfacing with hardware accelerators

A computer-implemented method of mixed-precision deep learning with multi-memristive synapses may be provided. The method comprises representing, each synapse of an artificial neural network by a combination of a plurality of memristive devices, wherein each of the plurality of memristive devices of each of the synapses contributes to an overall synaptic weight with a related device significance, accumulating a weight gradient &#x394;W for each synapse in a high-precision variable, and performing a weight update to one of the synapses using an arbitration scheme for selecting a respective memristive device, according to which a threshold value related to the high-precision variable for performing the weight update is set according to the device significance of the respective memristive device selected by the arbitration schema.

Mixed-precision deep-learning with multi-memristive devices

Artificial neuron apparatus includes first and second resistive memory cells. The first resistive memory cell is connected in first circuitry having a first input and output. The second resistive memory cell is connected in second circuitry having a second input and output. The first and second circuitry are operable in alternating read and write phases to apply a programming current to their respective memory cells on receipt of excitatory and inhibitory neuron input signals, respectively. During the write phase, resistance of the respective cells is changed in response to successive excitatory and inhibitory neuron input signals. During the read phase, a read current is applied to their respective cells to produce first and second measurement signals, respectively. An output circuit connected to the first and second outputs produces a neuron output signal at a neuron output when a difference between the first and second measurement signals traverses a threshold.

Artificial neuron apparatus

A resistive memory cell is connected in circuitry which has a first input terminal for applying neuron input signals including a read portion and a write portion. The circuitry includes a read circuit producing a read signal dependent on resistance of the memory cell, and an output terminal providing a neuron output signal, dependent on the read signal in a first state of the memory cell. The circuitry also includes a storage circuit storing a measurement signal dependent on the read signal, and a switch set operable to supply the read signal to the storage circuit during application of the read portion of each neuron input signal to the memory cell, and, after application of the read portion, to apply the measurement signal in the apparatus to enable resetting of the memory cell to a second state.

Growing community of inventors

Zurich, Switzerland

Evangelos Stavros Eleftheriou