Micron Technology Incorporated

Aliasger Tayeb Zaidy

Kunal R Parekh

Glen Earl Hush

Sean Stephen Eilert

Eugenio Culurciello

David Andrew Roberts

Andre Xian Ming Chang

Marko Vitez

Patrick Michael Sheridan

Lukasz Burzawa

Jaime Cummins

Patrick Alan Estep

Michael Cody Glapa

Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory (RAM). A compiler has an artificial neural network configured to identify an optimized compilation option for an artificial neural network to be compiled by the compiler and/or for a hardware platform of Deep Learning Accelerators. The artificial neural network of the compiler can be trained via machine learning to identify the optimized compilation option based on the features of the artificial neural network to be compiled and/or features of the hardware platform on which the compiler output will be executed.

Compiler with an artificial neural network to optimize instructions generated for execution on a deep learning accelerator of artificial neural networks

Disclosed in some examples are improved address prediction and memory preloading that leverages next-delta prediction and/or far-delta prediction for scheduling using a DNN. Previous memory access sequence data that identify one or more memory addresses previously accessed by one or more processors of a system may be processed and then converted into a sequence of delta values. The sequence of delta values are then mapped to one or more classes that are then input to a DNN. The DNN then outputs a predicted future class identifier sequence that represents addresses that the DNN predicts will be accessed by the processor in the future. The predicted future class identifier sequence is then converted back to a predicted delta value sequence and back into a set of one or more predicted addresses.

Delta predictions for page scheduling

A wafer-on-wafer bonded memory and logic device can enable high bandwidth transmission of data directly between a memory die and a logic die. Memory devices can be formed on a first wafer. First metal pads can be formed on the first wafer and coupled to the memory devices. The memory devices can be tested via the first metal pads. The first metal pads can be removed from the first wafer. Subsequently, second metal pads on the first wafer can be bonded, via a wafer-on-wafer bonding process, to third metal pads on a second wafer. Each memory device on the first wafer can be aligned with and coupled to a respective logic device on the second wafer.

Testing memory of wafer-on-wafer bonded memory and logic

A memory device includes a memory die bonded to a logic die. A logic die that is bonded to a memory die via a wafer-on-wafer bonding process can receive signals indicative of input data from a global data bus of the memory die and through a bond of the logic die and memory die. The logic die can also receive signals indicative of kernel data from local input/output (LIO) lines of the memory die and through the bond. The logic die can perform a plurality of operations at a plurality of vector-vector (VV) units utilizing the signals indicative of input data and the signals indicative of kernel data.

Signal routing between memory die and logic die for performing operations

A memory device includes a memory die bonded to a logic die via a wafer-on-wafer bond. A controller of the memory device that is coupled to the memory die can activate a row of the memory die. Responsive to activating the row, a sense amplifier stripe of the memory die can latch a first plurality of signals. A transceiver can route a second plurality of signals from the sense amplifier stripe to the logic die.

Signal routing between memory die and logic die

Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory. A computing device running a compiler can interact and/or probe an integrated circuit device to identify hardware characteristics of the integrated circuit device in performing matrix computations. The compiler can generate and optimize a result of compilation from a description of an artificial neural network based at least in part on the hardware characteristics of the integrated circuit device. The result of compilation can include first data representative of parameters of the artificial neural network and second data representative of instructions executable by the integrated circuit device to generate an output of the artificial neural network based on the first data and an input to the artificial neural network.

Discovery of hardware characteristics of deep learning accelerators for optimization via compiler

A memory device includes an array of memory cells configured on a die or chip and coupled to sense lines and access lines of the die or chip and a respective sense amplifier configured on the die or chip coupled to each of the sense lines. Each of a plurality of subsets of the sense lines is coupled to a respective local input/output (I/O) line on the die or chip for communication of data on the die or chip and a respective transceiver associated with the respective local I/O line, the respective transceiver configured to enable communication of the data to one or more device off the die or chip.

Memory device for wafer-on-wafer formed memory and logic

A memory device includes a memory die bonded to a logic die. A logic die that is bonded to a memory die via a wafer-on-wafer bonding process can receive signals indicative of input data from a global data bus of the memory die and through a bond of the logic die and memory die. The logic die can also receive signals indicative of kernel data from local input/output (LIO) lines of the memory die and through the bond. The logic die can perform a plurality of operations at a plurality of vector-vector (VV) units utilizing the signals indicative of input data and the signals indicative of kernel data. The inputs and the outputs to the VV units can be configured based on a mode of the logic die.

Signal routing between memory die and logic die for mode based operations

Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, the accelerator can have processing units to perform at least matrix computations of an artificial neural network via execution of instructions. The processing units have a local memory store operands of the instructions. The accelerator can access a random access memory via a system buffer, or without going through the system buffer. A fetch instruction can request an item, available at a memory address in the random access memory, to be loaded into the local memory at a local address. The fetch instruction can include a hint for the caching of the item in the system buffer. During execution of the instruction, the hint can be used to determine whether to load the item through the system buffer or to bypass the system buffer in loading the item.

Caching techniques for deep learning accelerator

A wafer-on-wafer formed memory and logic device can enable high bandwidth transmission of data directly between a memory die and a logic die. A logic die that is bonded to a memory die via a wafer-on-wafer bonding process can receive signals indicative of a genetic sequence from the memory die and through a wafer-on-wafer bond. The logic die can also perform a genome annotation lotic operation to attach biological information to the genetic sequence. An annotated genetic sequence can be provided as an output.

Wafer-on-wafer formed memory and logic for genomic annotations

A method of compiling neural network code to executable instructions for execution by a computational acceleration system having a memory circuit and one or more acceleration circuits having a maps data buffer and a kernel data buffer is disclosed, such as for execution by an inference engine circuit architecture which includes a matrix-matrix (MM) accelerator circuit having multiple operating modes to provide a complete matrix multiplication. A representative compiling method includes generating a list of neural network layer model objects; fusing available functions and layers in the list; selecting a cooperative mode, an independent mode, or a combined cooperative and independent mode for execution; selecting a data movement mode and an ordering of computations which reduces usage of the memory circuit; generating an ordered sequence of load objects, compute objects, and store objects; and converting the ordered sequence of load objects, compute objects, and store objects into the executable instructions.

Deep neural networks compiler for a trace-based accelerator

Various embodiments provide for one or more processor instructions and memory instructions that enable a memory sub-system to predict a schedule for migrating data between memory devices, which can be part of a memory sub-system.

Data migration schedule prediction using machine learning

An inference engine circuit architecture is disclosed which includes a matrix-matrix (MM) processor circuit and a MM accelerator circuit having multiple operating modes to provide a complete matrix multiplication. A representative MM accelerator circuit includes a first buffer circuit storing maps data; a first data network; multiple second buffer circuits each storing different kernel data; multiple second, serial data networks, with each coupled to a corresponding second buffer circuit; and a plurality of vector-vector (VV) acceleration circuits arranged in a plurality of arrays. Each VV acceleration circuit includes multiply and accumulate circuits; a shift register; a control multiplexer to provide a selected output, in response to a mode control word, of a bias parameter or a first accumulation sum; and a second adder circuit which adds the multiplicative product to the bias parameter or to the first accumulation sum to generate a second or next accumulation sum.

Growing community of inventors

Seattle, WA, United States of America

Aliasger Tayeb Zaidy