Memryx Incorporated

University of Michigan

Chester Liu

Wei Lu

Mohammed Zidan

Zhengya Zhang

Jacob Christopher Botimer

Timothy Wesley

Timothy Alan Wesley

Fan-hsuan Meng

Phil Knag

Thomas Chen

Ching-En Lee

Ching-Yu Ko

A memory processing unit (MPU) can include a first memory including a plurality of memory regions and a plurality of processing regions. One or more of the plurality of memory regions can be configured in a corresponding pluralities of memory blocks. The memory blocks can be configured to store weights in a selected one of a plurality of bit precisions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of core groups, wherein the core groups include one or more compute cores.

Core group memory processing with multi-precision weight packing

A processing unit can include a plurality of chiplets coupled in a cascade topology by a plurality of interfaces. A set of the plurality of cascade coupled chiplets can be configured to execute a plurality of layers or blocks of layers of an artificial intelligence model. The set of cascade coupled chiplets can also be configured with parameter data of corresponding ones of the plurality of layers or blocks of layers of the artificial intelligence model.

Chiplet based artificial intelligence accelerators and configuration methods

A memory processing unit (MPU) can include a first memory, a second memory, a plurality of processing regions and control logic. The first memory can include a plurality of regions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of compute cores. The second memory can be coupled to the plurality of processing regions. The control logic can configure data flow between compute cores of one or more of the processing regions and corresponding adjacent regions of the first memory. The control logic can also configure data flow between the second memory and the compute cores of one or more of the processing regions. The control logic can also configure data flow between compute cores within one or more respective ones of the processing regions. The control logic can also configure array data for storage memory of the MPU.

Memory processing unit core architectures

A memory processing unit (MPU) can include a first memory, a second memory, a plurality of processing regions and control logic. The first memory can include a plurality of regions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of compute cores. The second memory can be coupled to the plurality of processing regions. The control logic can configure data flow between compute cores of one or more of the processing regions and corresponding adjacent regions of the first memory. The control logic can also configure data flow between the second memory and the compute cores of one or more of the processing regions. The control logic can also configure data flow between compute cores within one or more respective ones of the processing regions.

Inter-layer communication techniques for memory processing unit architectures

Techniques for computing matrix convolutions in a plurality of multiply and accumulate units including data reuse of adjacent values. The data reuse can include reading a current value of the first matrix in from memory for concurrent use by the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory to a serial shift buffer coupled to the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory for concurrent use by the plurality of multiply and accumulate units.

Matrix data reuse techniques in multiply and accumulate units of processing system

Memory processing unit architectures and configurations

A memory processing unit (MPU) configuration method can include mapping operations of one or more neural network models to sets of cores in a plurality of processing regions. In addition, dataflow of the one or more neural network models can be mapped to the sets of cores in the plurality of processing regions. Furthermore, configuration information can be generated based on the mapping of the operations of the one or more neural network models to the set of cores in the plurality of processing regions and the mapping of dataflow of the one or more neural network models to the sets of cores in the plurality of processing regions. The method can be implemented by generating an initial graph from a neural network model. A mapping graph can then be generated from the final graph. One or more configuration files can then be generated from the mapping graph.

Memory processing unit architecture mapping techniques

A monolithic integrated circuit (IC) including one or more compute circuitry, one or more non-volatile memory circuits, one or more communication channels and one or more communication interface. The one or more communication channels can communicatively couple the one or more compute circuitry, the one or more non-volatile memory circuits and the one or more communication interface together. The one or more communication interfaces can communicatively couple one or more circuits of the monolithic integrated circuit to one or more circuits external to the monolithic integrated circuit.

Non-volatile memory based processors and dataflow techniques

A memory processing unit architecture can include a plurality of memory regions and a plurality of processing regions interleaved between the plurality of memory regions. The plurality of processing regions can be configured to perform computation functions of a model such as an artificial neural network. Data can be transferred between the computation functions in respective memory processing regions. In addition, the memory regions can be utilized to transfer data between a computation function in one processing region and a computation function in another processing region adjacent to the given memory region.

Memory processing unit architecture

A sparse video inference chip is designed to extract spatio-temporal features from videos for action classification and motion tracking. The core is a sparse video inference processor that implements recurrent neural network in three layers of processing. High sparsity is enforced in each layer of processing, reducing the complexity by two orders of magnitude and allowing all multiply-accumulates (MAC) to be replaced by select-accumulates (SA). The design is demonstrated in a 3.98 mm2 40 nm CMOS chip with an Open-RISC processor providing software-defined control and classification.

Sparse video inference processor for action classification and motion tracking

An information processor is provided that includes an inference module configured to extract a subset of data from information in an input and a classification module configured to classify the information in the input based on the extracted data. The inference module includes a first plurality of convolvers acting in parallel to apply each of N1 convolution kernels to each of N2 portions of the input image in order to generate an interim sparse representation of the input and a second plurality of convolvers acting in parallel to apply each of N3 convolution kernels to each of N4 portions of the interim sparse representation to generate a final sparse representation containing the extracted data. In order to take advantage of sparsity in the interim sparse representation, N3 is greater than N4 to parallelize processing in a non-sparse dimension and/or the second plurality of convolvers comprise sparse convolvers.

Sparse neuromorphic processor

A memory processing unit can be configured to compute partial products between one or more elements of a first matrix stored in a given row of a memory cell array and sequential bits of one or more elements of a second matrix. The partial products can be calculated first sequentially across the set of rows and second sequentially across the bit positions of the elements of the second matrix. Alternatively, the partial products can be calculated first sequentially across the bit positions of the elements of the second matrix first and second sequentially across the set of rows. The partial products for each column of elements can be accumulated and bit shifted to compute the dot product of the first and second matrix.

Memory processing units and methods of computing dot products

A memory processing unit can be configured to compute partial products between one or more elements of a first matrix stored in a first storage location and sequential bits of one or more elements of a second matrix stored in a second storage location. The partial products can be calculated utilizing zero bit skipping to increase throughput and or reduce energy consumption. The partial products for each column of elements can be accumulated and bit shifted to compute the dot product of the first and second matrix.

Growing community of inventors

Ann Arbor, MI, United States of America

Chester Liu