International Business Machines Corporation

Purdue Research Foundation

Shubham Jain

Swagath Venkataramani

Vijayalakshmi Srinivasan

Leland Chang

Geoffrey W Burr

Anand Raghunathan

Yasuteru Kohda

Pritish Narayanan

Milos Stanisavljevic

HsinYu Tsai

Sumeet Kumar Gupta

Kohji Hosokawa

Kaushik Roy

Jungwook Choi

Sunil K Shukla

Ashish Ranjan

Sanchari Sen

Martin A Lutz

Sarada Krithivasan

Arrays of neural cores are provided. Each neural core comprises ordered input wires ordered output wires, and synapses, each of the synapses operatively coupled to one of the input wires and one of the output wires. A plurality of signal wires is provided. At least one of the signal wires is disposed along each dimension of the array of neural cores. A plurality of routers is provided, each of which is operatively coupled to one of the neural cores and to at least one of the signal wires along each of the dimensions of the array of neural cores. Each of the routers selectively routes a signal from the at least one signal wire to its coupled neural core. Each of the routers selectively routes a signal from its coupled neural core to the at least one signal wire. The routers segment the ordered input wires and the ordered output wires into segments and independently routes the signals of each segment.

Implicit vector concatenation within 2D mesh routing

Embodiments are provided for efficient realization of memory-bound operations in a computing system by a processor. Data may be read from and written to a memory at a granular level using a stickification operation. One or more regions of activation and weight tensor data on the memory may be annotated by coupling the stickification operation with padding.

Stickification using anywhere padding to accelerate data manipulation

Embodiments disclosed herein include a compute in-memory (CIM) accelerator architecture for deep neural network (DNN). The CIM accelerator architecture may include a first analog fabric engine having a plurality of compute in-memory (CIM) analog tiles. Each CIM analog tile may be configured to store a matrix of weight operands producing a vector of outputs from a vector of inputs, and perform in-memory computations. The first analog fabric may also include a plurality of compute cores. Each CIM analog tile and each compute core may include a microcontroller configured to execute a set of instructions. The first analog fabric may also include on-chip interconnects communicatively connecting all CIM analog tiles in the plurality of CIM analog tile to the compute cores.

Two-dimensional mesh for compute-in-memory accelerator architecture

Efficient data layout and alignment techniques for effectively executing AI workloads in wide-vector accelerator systems are provided. In one aspect, a method for processing AI workloads includes: logically dividing a data vector into a hierarchy of segments and sub-segments with each of the segments including more than one of the sub-segments, wherein each of the sub-segments includes words, and each of the words includes data-bits; and physically mapping the data-bits such that the words belonging to a same given one of the sub-segments are mapped contiguously across all of the segments. An AI accelerator system is also provided.

Efficient data layout and alignment for wide-vector accelerator systems

A system comprises an analog resistive processing unit (RPU) system, and one or more processors. The analog RPU system comprises an array of RPU cells. The one or more processors are configured to: configure the analog RPU system to implement a convolutional neural network comprising a convolutional layer comprising at least one kernel matrix; program the at least one array of RPU cells to store a transformed kernel matrix which is generated by applying a first transformation process to the kernel matrix using a first predefined transformation matrix; and utilize the analog RPU system to perform an analog convolution operation by performing analog matrix-vector multiplication operations using the transformed kernel matrix and input vectors of a transformed data matrix, to thereby generate a transformed convolution output matrix, wherein the transformed data matrix is generated by applying a second transformation process to a data matrix using a second predefined transformation matrix.

Training convolution neural network on analog resistive processing unit system

A compensated deep neural network (compensated-DNN) is provided. A first vector having a set of components and a second vector having a set of corresponding components are received. A component of the first vector includes a first quantized value and a first compensation instruction, and a corresponding component of the second vector includes a second quantized value and a second compensation instruction. The first quantized value is multiplied with the second quantized value to compute a raw product value. The raw product value is compensated for a quantization error according to the first and second compensation instructions to produce a compensated product value. The compensated product value is added into an accumulated value for the dot product. The accumulated value is converted into an output vector of the dot product. The output vector includes an output quantized value and an output compensation instruction.

Low precision deep neural network enabled by compensation instructions

An efficient pipelined implementation of digital scaling, offset and aggregation operation supports element-by-element programmable scale and offset factors. The method includes time-multiplexed parallel pipelining of a plurality of digital data words, each of the plurality of digital data words encoding an N-bit signed integer, from one of a plurality of receive-registers through a datapath that can either (1) store the plurality of digital data words directly in a dedicated first memory, (2) store the plurality of digital data words directly in a dedicated second memory, or (3) direct the plurality of digital data words into a parallel set of fused-multiply-add units. The method further includes multiplying each digital data word by a corresponding data-word retrieved from the dedicated first memory to form product data words and adding the product data words to a corresponding data-word retrieved from the dedicated second memory to form an output sum-and-product data words.

Special-purpose digital-compute hardware for efficient element-wise aggregation, scaling and offset

A circuit of cells used as a memory array and capable of in-memory arithmetic which includes a plurality of signed ternary processing, each signed ternary processing cell includes a first memory cell, adapted to hold a first digital value, a second memory cell, adapted to hold a second digital value, wherein a binary combination of the first digital value and the second digital value establishes a first signed ternary operand, a signed ternary input forming a second signed ternary operand, and a signed ternary output, wherein the signed ternary output represents a signed multiplication of the first signed ternary operand and the second signed ternary operand, a sense circuit adapted to output a subtraction result.

Ternary in-memory accelerator

Embodiments for providing enhanced multicast data transfer for ring topology based artificial intelligence systems are disclosed. Multicast data is sent to a plurality of disjointed cores in a multicast group according to a first multicast mode, a second multicast mode, or a third multicast mode, where the first multicast mode sends a first half the multicast data on first multicast ring and a second half on a second multicast ring, the second multicast mode sends the multicast data on either the first multicast ring or the second multicast ring, and the third multicast mode replicates the multicast data and sends the multicast data to both the first multicast ring and the second multicast ring.

Programmable multicast protocol for ring-topology based artificial intelligence systems

A ternary processing cell used as a memory cell and capable of in-memory arithmetic is disclosed which includes a first memory cell, adapted to hold a first digital value, a second memory cell, adapted to hold a second digital value, wherein a binary combination of the first digital value and the second digital value establishes a first ternary operand, a ternary input establishing a second ternary operand, and a ternary output, wherein the ternary output represents a multiplication of the first ternary operand and the second ternary operand.

A method is provided for forming a Deep Neural Network (DNN). The method includes quantizing deep learning data structures of the DNN into at least two modes using at least two scale factors, respectively. Each of the at least two modes corresponds to a respective one of the at least two scale factors. The method further includes identifying which of the at least two scale factors to use for a given one of the data structures based on a data distribution of the given one of the data structures. The quantizing step includes identifying when a tail of the given one of the data structures starts by (i) building a histogram of values in the given one of the data structures using successive bins; (ii) identifying a ratio of density between the successive bins; and (iii) checking whether the ratio of density is greater than a ratio of density threshold.

Bi-scaled deep neural networks

A memory capable of carrying out compute-in-memory (CiM) operations is disclosed. The memory includes a matrix of bit cells having a plurality of bit cells along one or more rows and a plurality of bit cells along one or more columns, each bit cell having a value stored therein, an address decoder configured to receive addresses and activate two or more of the rows associated with the addresses, and a sensing circuit coupled to each column of bit cells, and configured to provide two or more outputs, wherein each output is associated with at least one compute operation performed on values stored in the bit cells in the column.

Growing community of inventors

West Lafayette, IN, United States of America

Shubham Jain