Memryx Incorporated

University of Michigan

Zhengya Zhang

Chester Liu

Wei Lu

Mohammed Zidan

Jacob Christopher Botimer

Fan-hsuan Meng

Timothy Alan Wesley

Timothy Wesley

Thomas Chen

Chia-Hsiang Chen

Phil Knag

Ching-En Lee

Shiming Song

Ching-Yu Ko

A processing unit can include a plurality of chiplets coupled in a cascade topology by a plurality of interfaces. A set of the plurality of cascade coupled chiplets can be configured to execute a plurality of layers or blocks of layers of an artificial intelligence model. The set of cascade coupled chiplets can also be configured with parameter data of corresponding ones of the plurality of layers or blocks of layers of the artificial intelligence model.

Chiplet based artificial intelligence accelerators and configuration methods

Techniques for computing matrix convolutions in a plurality of multiply and accumulate units including data reuse of adjacent values. The data reuse can include reading a current value of the first matrix in from memory for concurrent use by the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory to a serial shift buffer coupled to the plurality of multiply and accumulate units. The data reuse can also include reading a current value of the second matrix in from memory for concurrent use by the plurality of multiply and accumulate units.

Matrix data reuse techniques in multiply and accumulate units of processing system

A memory processing unit (MPU) can include a first memory, a second memory, a plurality of processing regions and control logic. The first memory can include a plurality of regions. The plurality of processing regions can be interleaved between the plurality of regions of the first memory. The processing regions can include a plurality of compute cores. The second memory can be coupled to the plurality of processing regions. The control logic can configure data flow between compute cores of one or more of the processing regions and corresponding adjacent regions of the first memory. The control logic can also configure data flow between the second memory and the compute cores of one or more of the processing regions. The control logic can also configure data flow between compute cores within one or more respective ones of the processing regions.

Memory processing unit architectures and configurations

A matrix multiplication engine can include a plurality of processing elements configured to compute a matrix dot product as a summation of a sequence of vector-vector outer-products.

Matrix multiplication engines

A monolithic integrated circuit (IC) including one or more compute circuitry, one or more non-volatile memory circuits, one or more communication channels and one or more communication interface. The one or more communication channels can communicatively couple the one or more compute circuitry, the one or more non-volatile memory circuits and the one or more communication interface together. The one or more communication interfaces can communicatively couple one or more circuits of the monolithic integrated circuit to one or more circuits external to the monolithic integrated circuit.

Non-volatile memory based processors and dataflow techniques

A memory processing unit architecture can include a plurality of memory regions and a plurality of processing regions interleaved between the plurality of memory regions. The plurality of processing regions can be configured to perform computation functions of a model such as an artificial neural network. Data can be transferred between the computation functions in respective memory processing regions. In addition, the memory regions can be utilized to transfer data between a computation function in one processing region and a computation function in another processing region adjacent to the given memory region.

Memory processing unit architecture

Many signal processing, machine learning and scientific computing applications require a large number of multiply-accumulate (MAC) operations. This type of operation is demanding in both computation and memory. Process in memory has been proposed as a new technique that computes directly on a large array of data in place, to eliminate expensive data movement overhead. To enable parallel multi-bit MAC operations, both width- and level-modulating memory word lines are applied. To improve performance and provide tolerance against process-voltage-temperature variations, a delay-locked loop is used to generate fine unit pulses for driving memory word lines and a dual-ramp Single-slope ADC is used to convert bit line outputs. The concept is prototyped in a 180 nm CMOS test chip made of four 320&#xd7;64 compute-SRAMs, each supporting 128&#xd7; parallel 5 b&#xd7;5 b MACs with 32 5 b output ADCs and consuming 16.6 mW at 200 MHz.

SRAM-based process in memory system

A sparse video inference chip is designed to extract spatio-temporal features from videos for action classification and motion tracking. The core is a sparse video inference processor that implements recurrent neural network in three layers of processing. High sparsity is enforced in each layer of processing, reducing the complexity by two orders of magnitude and allowing all multiply-accumulates (MAC) to be replaced by select-accumulates (SA). The design is demonstrated in a 3.98 mm2 40 nm CMOS chip with an Open-RISC processor providing software-defined control and classification.

Sparse video inference processor for action classification and motion tracking

An information processor is provided that includes an inference module configured to extract a subset of data from information in an input and a classification module configured to classify the information in the input based on the extracted data. The inference module includes a first plurality of convolvers acting in parallel to apply each of N1 convolution kernels to each of N2 portions of the input image in order to generate an interim sparse representation of the input and a second plurality of convolvers acting in parallel to apply each of N3 convolution kernels to each of N4 portions of the interim sparse representation to generate a final sparse representation containing the extracted data. In order to take advantage of sparsity in the interim sparse representation, N3 is greater than N4 to parallelize processing in a non-sparse dimension and/or the second plurality of convolvers comprise sparse convolvers.

Sparse neuromorphic processor

A memory processing unit can be configured to compute partial products between one or more elements of a first matrix stored in a given row of a memory cell array and sequential bits of one or more elements of a second matrix. The partial products can be calculated first sequentially across the set of rows and second sequentially across the bit positions of the elements of the second matrix. Alternatively, the partial products can be calculated first sequentially across the bit positions of the elements of the second matrix first and second sequentially across the set of rows. The partial products for each column of elements can be accumulated and bit shifted to compute the dot product of the first and second matrix.

Memory processing units and methods of computing dot products

A memory processing unit can be configured to compute partial products between one or more elements of a first matrix stored in a first storage location and sequential bits of one or more elements of a second matrix stored in a second storage location. The partial products can be calculated utilizing zero bit skipping to increase throughput and or reduce energy consumption. The partial products for each column of elements can be accumulated and bit shifted to compute the dot product of the first and second matrix.

Memory processing units and methods of computing DOT products including zero bit skipping

A reciprocal unit for computing an estimated reciprocal of a number represented by a bit string. The unit comprises a first lookup table configured to receive one or more of the bits in the bit string and to output an initial estimate of the reciprocal of the number. The unit further comprises a second lookup table configured to receive one or more of the bits in the bit string and to output the square of the initial estimate of the reciprocal of the number. The unit still further comprises a multiplier circuit configured to multiply the square of the initial estimate by the number, and an adder-subtractor circuit for subtracting the product of the multiplication from a scaled value of the initial estimate to determine a final estimate of the reciprocal of the number.

Reciprocal unit

A nonbinary iterative detector-decoder (IDD) system. The IDD system comprises a detector, a decoder; and a nonbinary interface electrically connected between the detector and decoder. The interface is operative to convert a soft symbol and variance that is output by the detector into a corresponding nonbinary log likelihood ratio (LLR) vector that comprises one or more nonbinary LLRs, and to provide the LLR vector to the decoder. The interface is further configured to convert a nonbinary LLR vector comprised of one or more nonbinary LLRs that is output by the decoder into a corresponding soft symbol and variance, and to provide the soft symbol and variance to the detector.

Growing community of inventors

Ann Arbor, MI, United States of America

Zhengya Zhang