3d hybrid bonding 3d memory devices with npu/cpu for ai inference application

Abstract

An AI inference platform comprises a logic die including an array of AI processing elements. Each AI processing element including an activation memory storing activation data for use in neural network computations. The platform includes a memory die that includes an array of 3D memory cells and a page buffer that facilitates storage and retrieval of neural network weights for use in neural network computations. A plurality of vertical connections can directly connect AI processing elements in the logic die and page buffers of corresponding ones of the memory cells in the memory die, enabling storage or retrieval of a neural network weight to and from a particular page buffer of a corresponding 3D memory cell for use in neural network computations conducted by a corresponding AI processing element in the logic die.