Memryx Incorporated

University of Michigan

Jacob Christopher Botimer

Wei Lu

Mohammed Zidan

Chester Liu

Zhengya Zhang

Timothy Wesley

Timothy Alan Wesley

Fan-hsuan Meng

Thomas Chen

Ching-Yu Ko

Shiming Song

A memory processing unit (MPU) can include a first memory including a plurality of memory regions and a plurality of processing regions. One or more of the plurality of memory regions can be configured in a corresponding pluralities of memory blocks. The memory blocks can be configured to store weights in a selected one of a plurality of bit precisions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of core groups, wherein the core groups include one or more compute cores.

Core group memory processing with multi-precision weight packing

A processing unit can include a plurality of chiplets coupled in a cascade topology by a plurality of interfaces. A set of the plurality of cascade coupled chiplets can be configured to execute a plurality of layers or blocks of layers of an artificial intelligence model. The set of cascade coupled chiplets can also be configured with parameter data of corresponding ones of the plurality of layers or blocks of layers of the artificial intelligence model.

Chiplet based artificial intelligence accelerators and configuration methods

A memory processing unit (MPU) can include a first memory, a second memory, a plurality of processing regions and control logic. The first memory can include a plurality of regions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of compute cores. The second memory can be coupled to the plurality of processing regions. The control logic can configure data flow between compute cores of one or more of the processing regions and corresponding adjacent regions of the first memory. The control logic can also configure data flow between the second memory and the compute cores of one or more of the processing regions. The control logic can also configure data flow between compute cores within one or more respective ones of the processing regions. The control logic can also configure array data for storage memory of the MPU.

Memory processing unit core architectures

A memory processing unit (MPU) can include a first memory, a second memory, a plurality of processing regions and control logic. The first memory can include a plurality of regions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of compute cores. The second memory can be coupled to the plurality of processing regions. The control logic can configure data flow between compute cores of one or more of the processing regions and corresponding adjacent regions of the first memory. The control logic can also configure data flow between the second memory and the compute cores of one or more of the processing regions. The control logic can also configure data flow between compute cores within one or more respective ones of the processing regions.

Inter-layer communication techniques for memory processing unit architectures

Techniques for computing matrix convolutions in a plurality of multiply and accumulate units including data reuse of adjacent values. The data reuse can include reading a current value of the first matrix in from memory for concurrent use by the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory to a serial shift buffer coupled to the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory for concurrent use by the plurality of multiply and accumulate units.

Matrix data reuse techniques in multiply and accumulate units of processing system

Memory processing unit architectures and configurations

A memory processing unit (MPU) configuration method can include mapping operations of one or more neural network models to sets of cores in a plurality of processing regions. In addition, dataflow of the one or more neural network models can be mapped to the sets of cores in the plurality of processing regions. Furthermore, configuration information can be generated based on the mapping of the operations of the one or more neural network models to the set of cores in the plurality of processing regions and the mapping of dataflow of the one or more neural network models to the sets of cores in the plurality of processing regions. The method can be implemented by generating an initial graph from a neural network model. A mapping graph can then be generated from the final graph. One or more configuration files can then be generated from the mapping graph.

Memory processing unit architecture mapping techniques

A monolithic integrated circuit (IC) including one or more compute circuitry, one or more non-volatile memory circuits, one or more communication channels and one or more communication interface. The one or more communication channels can communicatively couple the one or more compute circuitry, the one or more non-volatile memory circuits and the one or more communication interface together. The one or more communication interfaces can communicatively couple one or more circuits of the monolithic integrated circuit to one or more circuits external to the monolithic integrated circuit.

Non-volatile memory based processors and dataflow techniques

A memory processing unit architecture can include a plurality of memory regions and a plurality of processing regions interleaved between the plurality of memory regions. The plurality of processing regions can be configured to perform computation functions of a model such as an artificial neural network. Data can be transferred between the computation functions in respective memory processing regions. In addition, the memory regions can be utilized to transfer data between a computation function in one processing region and a computation function in another processing region adjacent to the given memory region.

Memory processing unit architecture

Many signal processing, machine learning and scientific computing applications require a large number of multiply-accumulate (MAC) operations. This type of operation is demanding in both computation and memory. Process in memory has been proposed as a new technique that computes directly on a large array of data in place, to eliminate expensive data movement overhead. To enable parallel multi-bit MAC operations, both width- and level-modulating memory word lines are applied. To improve performance and provide tolerance against process-voltage-temperature variations, a delay-locked loop is used to generate fine unit pulses for driving memory word lines and a dual-ramp Single-slope ADC is used to convert bit line outputs. The concept is prototyped in a 180 nm CMOS test chip made of four 320&#xd7;64 compute-SRAMs, each supporting 128&#xd7; parallel 5 b&#xd7;5 b MACs with 32 5 b output ADCs and consuming 16.6 mW at 200 MHz.

Growing community of inventors

Ann Arbor, MI, United States of America

Jacob Christopher Botimer