Intel Corporation

University of Michigan

Phil Knag

Gregory Kengho Chen

Huseyin Ekin Sumbul

Ram K Krishnamurthy

Raghavan Kumar

Abhishek A Sharma

Sasikanth Manipatruni

Ian A Young

Amrita Mathuriya

Van H Le

Victor W Lee

Jack T Kavalieros

Hui Jae Yoo

Uygar E Avci

Noriyuki Sato

Nazila Haratipour

Sarah Atanasov

Ravi Pillarisetty

Huichu Liu

Zhengya Zhang

Chester Liu

Abhronil Sengupta

Ram Kumar Krishnamurthy

Deepak Vinayak Kadetotad

An apparatus is described. The apparatus includes a long short term memory (LSTM) circuit having a multiply accumulate circuit (MAC). The MAC circuit has circuitry to rely on a stored product term rather than explicitly perform a multiplication operation to determine the product term if an accumulation of differences between consecutive, preceding input values has not reached a threshold.

LSTM circuit with selective input computation

Examples herein relate to a memory device comprising an eDRAM memory cell, the eDRAM memory cell can include a write circuit formed at least partially over a storage cell and a read circuit formed at least partially under the storage cell; a compute near memory device bonded to the memory device; a processor; and an interface from the memory device to the processor. In some examples, circuitry is included to provide an output of the memory device to emulate output read rate of an SRAM memory device comprises one or more of: a controller, a multiplexer, or a register. Bonding of a surface of the memory device can be made to a compute near memory device or other circuitry. In some examples, a layer with read circuitry can be bonded to a layer with storage cells. Any layers can be bonded together using techniques described herein.

Compute near memory with backend memory

Various systems, devices, and methods for operating on a data sequence. A system includes a set of circuits that form an input layer to receive a data sequence; first hardware computing units to transform the data sequence, the first hardware computing units connected using a set of randomly selected weights, a first hardware computing unit to: receive an input from a second hardware computing unit, determine a weight of a connection between the first and second hardware computing units using an identifier of the second hardware computing unit and a fixed random weight generator, and operate on the input using the weight to determine a state of the first hardware computing unit; and second hardware computing units to operate on states of the first computing units to generate an output based on the data sequence.

Memoryless weight storage hardware for neural networks

Embodiments herein describe techniques for a semiconductor device including a RRAM memory cell. The RRAM memory cell includes a FinFET transistor and a RRAM storage cell. The FinFET transistor includes a fin structure on a substrate, where the fin structure includes a channel region, a source region, and a drain region. An epitaxial layer is around the source region or the drain region. A RRAM storage stack is wrapped around a surface of the epitaxial layer. The RRAM storage stack includes a resistive switching material layer in contact and wrapped around the surface of the epitaxial layer, and a contact electrode in contact and wrapped around a surface of the resistive switching material layer. The epitaxial layer, the resistive switching material layer, and the contact electrode form a RRAM storage cell. Other embodiments may be described and/or claimed.

FinFET transistor based resistive random access memory

An apparatus is described. The apparatus includes a compute-in-memory (CIM) circuit for implementing a neural network disposed on a semiconductor chip. The CIM circuit includes a mathematical computation circuit coupled to a memory array. The memory array includes an embedded dynamic random access memory (eDRAM) memory array. Another apparatus is described. The apparatus includes a compute-in-memory (CIM) circuit for implementing a neural network disposed on a semiconductor chip. The CIM circuit includes a mathematical computation circuit coupled to a memory array. The mathematical computation circuit includes a switched capacitor circuit. The switched capacitor circuit includes a back-end-of-line (BEOL) capacitor coupled to a thin film transistor within the metal/dielectric layers of the semiconductor chip. Another apparatus is described. The apparatus includes a compute-in-memory (CIM) circuit for implementing a neural network disposed on a semiconductor chip. The CIM circuit includes a mathematical computation circuit coupled to a memory array. The mathematical computation circuit includes an accumulation circuit. The accumulation circuit includes a ferroelectric BEOL capacitor to store a value to be accumulated with other values stored by other ferroelectric BEOL capacitors.

Applications of back-end-of-line (BEOL) capacitors in compute-in-memory (CIM) circuits

A compute near memory (CNM) convolution accelerator enables a convolutional neural network (CNN) to use dedicated acceleration to achieve efficient in-place convolution operations with less impact on memory and energy consumption. A 2D convolution operation is reformulated as 1D row-wise convolution. The 1D row-wise convolution enables the CNM convolution accelerator to process input activations row-by-row, while using the weights one-by-one. Lightweight access circuits provide the ability to stream both weights and input rows as vectors to MAC units, which in turn enables modules of the CNM convolution accelerator to implement convolution for both [1&#xd7;1] and chosen [n&#xd7;n] sized filters.

Compute near memory convolution accelerator

An apparatus is formed. The apparatus includes a stack of semiconductor chips. The stack of semiconductor chips includes a logic chip and a memory stack, wherein, the logic chip includes at least one of a GPU and CPU. The apparatus also includes a semiconductor chip substrate. The stack of semiconductor chips are mounted on the semiconductor chip substrate. At least one other logic chip is mounted on the semiconductor chip substrate. The semiconductor chip substrate includes wiring to interconnect the stack of semiconductor chips to the at least one other logic chip.

Multi-chip module having a stacked logic chip and memory stack

An apparatus is described. The apparatus includes a circuit to process a binary neural network. The circuit includes an array of processing cores, wherein, processing cores of the array of processing cores are to process different respective areas of a weight matrix of the binary neural network. The processing cores each include add circuitry to add only those weights of an i layer of the binary neural network that are to be effectively multiplied by a non zero nodal output of an i&#x2212;1 layer of the binary neural network.

Processor array for processing sparse binary neural networks

Examples described herein relate to a neural network whose weights from a matrix are selected from a set of weights stored in a memory on-chip with a processing engine for generating multiply and carry operations. The number of weights in the set of weights stored in the memory can be less than a number of weights in the matrix thereby reducing an amount of memory used to store weights in a matrix. The weights in the memory can be generated in training using gradients from back propagation. Weights in the memory can be selected using a tabulation hash calculation on entries in a table.

Reconfigurable memory compression techniques for deep neural networks

A DIMA semiconductor structure is disclosed. The DIMA semiconductor structure includes a frontend including a semiconductor substrate, a transistor switch of a memory cell coupled to the semiconductor substrate and a computation circuit on the periphery of the frontend coupled to the semiconductor substrate. Additionally, the DIMA includes a backend that includes an RRAM component of the memory cell that is coupled to the transistor switch.

Deep in memory architecture using resistive switches

Embodiments are directed to systems and methods of implementing an analog neural network using a pipelined SRAM architecture ('PISA') circuitry disposed in on-chip processor memory circuitry. The on-chip processor memory circuitry may include processor last level cache (LLC) circuitry. One or more physical parameters, such as a stored charge or voltage, may be used to permit the generation of an in-memory analog output using a SRAM array. The generation of an in-memory analog output using only word-line and bit-line capabilities beneficially increases the computational density of the PISA circuit without increasing power requirements. Thus, the systems and methods described herein beneficially leverage the existing capabilities of on-chip SRAM processor memory circuitry to perform a relatively large number of analog vector/tensor calculations associated with execution of a neural network, such as a recurrent neural network, without burdening the processor circuitry and without significant impact to the processor power requirements.

In-memory analog neural cache

The present disclosure is directed to systems and methods of implementing a neural network using in-memory, bit-serial, mathematical operations performed by a pipelined SRAM architecture (bit-serial PISA) circuitry disposed in on-chip processor memory circuitry. The on-chip processor memory circuitry may include processor last level cache (LLC) circuitry. The bit-serial PISA circuitry is coupled to PISA memory circuitry via a relatively high-bandwidth connection to beneficially facilitate the storage and retrieval of layer weights by the bit-serial PISA circuitry during execution. Direct memory access (DMA) circuitry transfers the neural network model and input data from system memory to the bit-serial PISA memory and also transfers output data from the PISA memory circuitry to system memory circuitry. Thus, the systems and methods described herein beneficially leverage the on-chip processor memory circuitry to perform a relatively large number of vector/tensor calculations without burdening the processor circuitry.

Low synch dedicated accelerator with in-memory computation capability

A memory circuit includes a number (X) of multiply-accumulate (MAC) circuits that are dynamically configurable. The MAC circuits can either compute an output based on computations of X elements of the input vector with the weight vector, or to compute the output based on computations of a single element of the input vector with the weight vector, with each element having a one bit or multibit length. A first memory can hold the input vector having a width of X elements and a second memory can store the weight vector. The MAC circuits include a MAC array on chip with the first memory.

Compute in/near memory (CIM) circuit architecture for unified matrix-matrix and matrix-vector computations

The present disclosure is directed to systems and methods of bit-serial, in-memory, execution of at least an nlayer of a multi-layer neural network in a first on-chip processor memory circuitry portion contemporaneous with prefetching and storing layer weights associated with the (n+1)layer of the multi-layer neural network in a second on-chip processor memory circuitry portion. The storage of layer weights in on-chip processor memory circuitry beneficially decreases the time required to transfer the layer weights upon execution of the (n+1)layer of the multi-layer neural network by the first on-chip processor memory circuitry portion. In addition, the on-chip processor memory circuitry may include a third on-chip processor memory circuitry portion used to store intermediate and/or final input/output values associated with one or more layers included in the multi-layer neural network.

Growing community of inventors

Hillsboro, OR, United States of America

Phil Knag