Methods and circuits for aggregating processing units and dynamically allocating memory

Abstract

An application-specific integrated circuit for an artificial neural network is integrated with a high-bandwidth memory. A processing die with tiled neural-network processing units is bonded to a stack of memory dies with memory banks laid out to establish relatively short connections to overlying processing units. The memory banks form vertical groups of banks for each overlying processing unit. A switch matrix on the processing die allows each processing unit to communicate with its vertical group of banks via a short, fast inter-die memory channel or with more remote groups of banks under neighboring processing units.