International Business Machines Corporation

Macronix International Co., Ltd.

University of Chicago

Qimonda North America Corporation

Polytechnic University of Turin

Geoffrey W Burr

Pritish Narayanan

HsinYu Tsai

Chung H Lam

Stephen Mark Rossnagel

Chandrasekharan Kothandaraman

Xiao Hu Liu

Kailash Gopalakrishnan

Robert L Wisnieff

Christy Sensenich Tyberg

Shubham Jain

Lawrence Alfred Clevenger

Hsiang-Lan Lung

Alejandro Gabriel Schrott

Nicholas Anthony Lanzillo

Masatoshi Ishii

Seiji Munetoh

Yi-Chou Chen

Kohji Hosokawa

Yasuteru Kohda

Milos Stanisavljevic

Stefano Ambrogio

Biswanath Senapati

Charles Mackin

Paul Michael Solomon

Thomas Happ

Matthew Joseph Breitwisch

Eric Andrew Joseph

Hemantha Kumar Wickramasinghe

Jan Boris Philipp

Jonathan Zanhong Sun

Hsiang Lan Lung

Mohit Bajaj

Kota V R M Murali

Simone Raoux

Rohit Sudhir Shenoy

Rajesh Sathiyanarayanan

Malte Johannes Rasch

Rajan K Pandey

Kumar R Virwani

Pierce I-Jen Chuang

Shih Hung Chen

Hans J Coufal

Yi Chou Chen

Irem Boybat Kara

Chieh Fang Chen

Robert Shelby

Giorgio Cristiano

Massimo Giordano

Carmelo Di Nolfo

Sebastian Kobras

Benjamin Killeen

Embodiments disclosed herein include a compute in-memory (CIM) accelerator architecture for deep neural network (DNN). The CIM accelerator architecture may include a first analog fabric engine having a plurality of compute in-memory (CIM) analog tiles. Each CIM analog tile may be configured to store a matrix of weight operands producing a vector of outputs from a vector of inputs, and perform in-memory computations. The first analog fabric may also include a plurality of compute cores. Each CIM analog tile and each compute core may include a microcontroller configured to execute a set of instructions. The first analog fabric may also include on-chip interconnects communicatively connecting all CIM analog tiles in the plurality of CIM analog tile to the compute cores.

Two-dimensional mesh for compute-in-memory accelerator architecture

Efficient data layout and alignment techniques for effectively executing AI workloads in wide-vector accelerator systems are provided. In one aspect, a method for processing AI workloads includes: logically dividing a data vector into a hierarchy of segments and sub-segments with each of the segments including more than one of the sub-segments, wherein each of the sub-segments includes words, and each of the words includes data-bits; and physically mapping the data-bits such that the words belonging to a same given one of the sub-segments are mapped contiguously across all of the segments. An AI accelerator system is also provided.

Efficient data layout and alignment for wide-vector accelerator systems

An IC memory device includes a substrate and an array of memory cells on the substrate. Each memory cell includes at least one memory cell transistor in a layer of the device adjacent to the substrate. In the same layer, the device also includes a plurality of shunt transistors. The device also includes a buried metal signal rail, which is disposed between the array of memory cells and the plurality of shunt transistors in a buried layer that is embedded into the substrate below the transistors. The device also includes single-layer vias, which are in same layer as the transistors and electrically connect the memory cell transistors to the shunt transistors through the buried metal signal rail.

Buried metal signal rail for memory arrays

A computer-implemented method, according to one embodiment, includes: causing a multi-bit input to be split into two or more chunks, where each of the two or more chunks include at least one individual bit. Each of the two or more chunks are also converted into a respective pulse width modulated signal, and a partial result is generated in digital form for each of the respective pulse width modulated signals. Each of the partial results are scaled by a respective significance factor corresponding to each of the two or more chunks, and the scaled partial results are also accumulated.

Split pulse width modulation to reduce crossbar array integration time

A memory cell in a backside of a wafer and methods of forming the memory cell are described. A buried metal structure can be formed through a frontside of a substrate. At least one device can be formed on the frontside of a substrate, where the at least one device can be connected to the buried metal structure in the substrate. A through silicon via (TSV) can be formed through a backside of the substrate, where the TSV can be connected to the buried metal structure. A memory cell can be formed on the backside of the substrate, where the memory cell can be connected to the TSV.

Memory cell in wafer backside

According to one or more embodiments, a computer implemented method for implementing a convolutional neural network (CNN) using a crosspoint array includes configuring the crosspoint array corresponding to a convolution layer in the CNN by storing one or more convolution kernels of the convolution layer in one or more crosspoint devices of the crosspoint array. The method further includes performing computations for the CNN via the crosspoint array by transmitting voltage pulses corresponding to a vector of input data of the convolution layer to the crosspoint array. Performing the CNN computations further includes outputting an electric current representative of performing a multiplication operation at a crosspoint device in the crosspoint array based on a weight value stored by the crosspoint device and the voltage pulses from the input data. Performing the CNN computations further includes passing the output electric current from the crosspoint device to a selected integrator.

Efficient processing of convolutional neural network layers using analog-memory-based hardware

Array-integrated upstream/downstream routers for circuit-switched parallel connectivity are provided. A system comprises an array of neural cores having at least one dimension, a plurality of signal wires, and a plurality of routers. Each neural core comprises a plurality of ordered input wires, a plurality of ordered output wires, and a plurality of synapses, each synapse operatively coupled to one of the plurality of input wires and one of the plurality of output wires. The plurality of signal wires are disposed along each dimension of the array of neural cores. Each router is operatively coupled to one of the plurality of neural cores and to at least one signal wire along each dimension of the array of neural cores. Each of the plurality of routers is adapted to selectively route a signal from the at least one signal wire to its coupled neural core. Each of the plurality of routers is adapted to selectively route a signal from its coupled neural core to the at least one signal wire.

Array-integrated upstream/downstream router for circuit switched parallel connectivity

A computer implemented method for implementing a convolutional neural network (CNN) using a crosspoint array includes configuring the crosspoint array to implement a convolution layer by storing one or more weights in crosspoint devices of the array. The method further includes making multiple copies of the weights and training the CNN. Training the CNN includes mapping input data of the convolution layer to the crosspoint array in a row-by-row manner. Further the excitation is input in a row-by-row manner into the crosspoint array, thereby creating row-by-row forward output from the crosspoint array. Further, outputs from the crosspoint devices are stored to corresponding integrators. Errors in the outputs as compared to a desired output, from multiple rows are computed and back propagated in a row-by-row manner into the crosspoint array, the computed errors transmitted to a previous convolution layer.

Row-by-row convolutional neural network mapping for analog artificial intelligence network training

An efficient pipelined implementation of digital scaling, offset and aggregation operation supports element-by-element programmable scale and offset factors. The method includes time-multiplexed parallel pipelining of a plurality of digital data words, each of the plurality of digital data words encoding an N-bit signed integer, from one of a plurality of receive-registers through a datapath that can either (1) store the plurality of digital data words directly in a dedicated first memory, (2) store the plurality of digital data words directly in a dedicated second memory, or (3) direct the plurality of digital data words into a parallel set of fused-multiply-add units. The method further includes multiplying each digital data word by a corresponding data-word retrieved from the dedicated first memory to form product data words and adding the product data words to a corresponding data-word retrieved from the dedicated second memory to form an output sum-and-product data words.

Special-purpose digital-compute hardware for efficient element-wise aggregation, scaling and offset

Computations in Artificial neural networks (ANNs) are accomplished using simple processing units, called neurons, with data embodied by the connections between neurons, called synapses, and by the strength of these connections, the synaptic weights. Crossbar arrays may be used to represent one layer of the ANN with Non-Volatile Memory (NVM) elements at each crosspoint, where the conductance of the NVM elements may be used to encode the synaptic weights, and a highly parallel current summation on the array achieves a weighted sum operation that is representative of the values of the output neurons. A method is outlined to transfer such neuron values from the outputs of one array to the inputs of a second array with no need for global clock synchronization, irrespective of the distances between the arrays, and to use such values at the next array, and/or to convert such values into digital bits at the next array.

System, method and article of manufacture for synchronization-free transmittal of neuron values in a hardware artificial neural networks

Artificial neural networks (ANNs) are a distributed computing model in which computation is accomplished with many simple processing units, called neurons, with data embodied by the connections between neurons, called synapses, and by the strength of these connections, the synaptic weights. An attractive implementation of ANNs uses the conductance of non-volatile memory (NVM) elements to record the synaptic weight, with the important multiply—accumulate step performed in place, at the data. In this application, the non-idealities in the response of the NVM such as nonlinearity, saturation, stochasticity and asymmetry in response to programming pulses lead to reduced network performance compared to an ideal network implementation. A method is shown that improves performance by periodically inverting the polarity of less-significant signed analog conductance-pairs within synaptic weights that are distributed across multiple conductances of varying significance, upon transfer of weight information between less-significant signed analog conductance-pairs to more-significant analog conductance-pairs.

Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries

Implementing a convolutional neural network (CNN) includes configuring a crosspoint array to implement a convolution layer in the CNN. Convolution kernels of the layer are stored in crosspoint devices of the array. Computations for the CNN are performed by iterating a set of operations for a predetermined number of times. The operations include transmitting voltage pulses corresponding to a subpart of a vector of input data to the crosspoint array. The voltage pulses generate electric currents that are representative of performing multiplication operations at the crosspoint device based on weight values stored at the crosspoint devices. A set of integrators accumulates an electric charge based on the output electric currents from the respective crosspoint devices. The crosspoint array outputs the accumulated charge after iterating for the predetermined number of times. The accumulated charge represents a multiply-add result of the vector of input data and the one or more convolution kernels.

Efficient tile mapping for row-by-row convolutional neural network mapping for analog artificial intelligence network inference

A computer-implemented method, according to one embodiment, includes: causing a first subset of pulse width modulators in a crossbar array of memory cells to apply respective pulses to the crossbar array together at a same start time and end the respective pulses according to a predetermined distribution of times correlated to stored pulse width data for each pulse width modulator. The method also includes causing a second subset of pulse width modulators in the crossbar array to apply pulses to the crossbar array according to the predetermined distribution of times correlated to stored pulse width data for each pulse width modulator and end the respective pulses together at a same end time.

Selective application of multiple pulse durations to crossbar arrays

Optimized synapses for neuromorphic arrays are provided. In various embodiments, first and second single-transistor current sources are electrically coupled in series. The first single-transistor current source is electrically coupled to both a first control circuit and second control circuit, free of any intervening logic gate between the first single-transistor current source and either one of the control circuits. The second single-transistor current source is electrically coupled to both the first control circuit and the second control circuit, free of any intervening logic gate between the second single-transistor current source and either one of the control circuits. A capacitor is electrically coupled to the first and second single-transistor current sources. A read circuit is electrically coupled to the capacitor. The first and second single-transistor current sources are adapted to charge the capacitor only when concurrently receiving a control signal from both the first and second control circuits. The first and second single-transistor current sources are adapted to discharge the capacitor only when concurrently receiving a control signal from both the first and second control circuit.

Competitive machine learning accuracy on neuromorphic arrays with non-ideal non-volatile memory devices

Growing community of inventors

San Jose, CA, United States of America

Geoffrey W Burr