Tilera Corporation

Arm Limited

Intel Corporation

Mellanox Technologies, Ltd.

Ezchip Technologies Ltd.

Mellanoz Technologeis, Ltd.

Mallanox Technologies Ltd.

Matthew Mattina

Anant Agarwal

Ian Rudolf Bratt

Paul Nicholas Whatmough

David M Wentzlaff

Carl Ramey

Zhi-Gang Liu

George Zacharias Chrysos

Jesse Garrett Beu

Chyi-Chang Miao

Zhigang Liu

Shidhartha Das

Glen Arnold Rosendale

Fernando Garcia Redondo

Dibakar Gope

Mark Jay Charney

Shubhendu Sekhar Mukherjee

Mark B Rosenbluth

Joel S Emer

Ravi Rajwar

David Hennah Mansell

Saurabh Pijuskumar Sinha

Bruce Edwards

Christopher J Jackson

Yungho Choi

Supreet Jeloka

Richard Schooler

Christopher D Metcalf

Liewei Bao

Patrick Robert Griffin

Kenneth M Steele

Shane Bell

John A Amann

Pritpal S Ahuja

Paul B Racunas

Mathew Hostetter

Andrew Carlson

Richard Conlin

Kevin Joyce

Christine Deignan

Antonio Juan-Hormigo

Ramon Matas-Navarro

Stephen Felix

John Fremont Brown, Iii

Ben Harrison

A system and method for multiplying matrices, and method for training a convolutional neural network (CNN), are provided. The system includes a processor and a matrix multiply accelerator (MMA). The processor is configured to generate, based on an input tensor, a number of basic block matrices, each basic block matrix including a number of elements; for each basic block matrix: prune, based on a sparsity value, the elements of the basic block matrix, generate a mask for the basic block matrix, each mask including a number of bits, each bit corresponding to a different element of the basic block matrix, and compress the basic block matrix to generate a compressed basic block matrix having fewer elements than the basic block matrix. The MMA is configured to multiply, based on the masks, the compressed basic block matrices and a weight matrix to generate an output matrix.

Activation compression method for deep learning acceleration

The present disclosure advantageously provides a system including a memory, a processor, and a circuitry to execute one or more mixed precision layers of an artificial neural network (ANN), each mixed precision layer including high-precision weight filters and low precision weight filters. The circuitry is configured to perform one or more calculations on an input feature map having a plurality of input channels (c) using the high precision weight filters to create a high precision output feature map having a first number of output channels (k), perform one or more calculations on the input feature map using the low precision weight filters to create a low precision output feature map having a second number of output channels (c−k), and concatenate the high precision output feature map and the low precision output feature map to create a unified output feature map having a plurality of output channels (c).

Hybrid filter banks for artificial neural networks

A system and method for multiplying matrices are provided. The system includes a processor coupled to a memory and a matrix multiply accelerator (MMA) coupled to the processor. The MMA is configured to multiply, based on a bitmap, a compressed first matrix and a second matrix to generate an output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the compressed first matrix and a jcolumn of the second matrix based on the bitmap. Or, the MMA is configured to multiply, based on the bitmap, the second matrix and the compressed first matrix and to generate the output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the second matrix and a jcolumn of the compressed first matrix based on the bitmap.

Time domain unrolling sparse matrix multiplication system and method

A method and apparatus for performing refactored multiply-and-accumulate operations is provided. A summing array includes a plurality of non-volatile memory elements arranged in columns. Each non-volatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on weights of a neural network. The summing array is configured to generate a summed signal for each column based, at least in part, on a plurality of input signals. A multiplying array is coupled to the summing array, and includes a plurality of non-volatile memory elements. Each non-volatile memory element in the multiplying array is programmed to a different conductance level based on the weights of the neural network. The multiplying array is configured to generate an output signal based, at least in part, on the summed signals from the summing array.

Refactoring mac operations

A multiply-accumulate method and architecture are disclosed. The architecture includes a plurality of networks of non-volatile memory elements arranged in tiled columns. Logic digitally modulates the equivalent conductance of individual networks among the plurality of networks to map the equivalent conductance of each individual network to a single weight within the neural network. A first partial selection of weights within the neural network is mapped into the equivalent conductances of the networks in the columns to enable the computation of multiply-and-accumulate operations by mixed-signal computation. The logic updates the mappings to select a second partial selection of weights to compute additional multiply-and-accumulate operations and repeats the mapping and computation operations until all computations for the neural network are completed.

Non-volatile memory-based compact mixed-signal multiply-accumulate engine

The present disclosure advantageously provides a matrix expansion unit that includes an input data selector, a first register set, a second register set, and an output data selector. The input data selector is configured to receive first matrix data in a columnwise format. The first register set is coupled to the input data selector, and includes a plurality of data selectors and a plurality of registers arranged in a first shift loop. The second register set is coupled to the data selector, and includes a plurality of data selectors and a plurality of registers arranged in a second shift loop. The output data selector is coupled to the first register set and the second register set, and is configured to output second matrix data in a rowwise format.

Hardware accelerator for IM2COL operation

Various implementations described herein are directed to a device having a multi-layered logic structure with a first logic layer and a second logic layer arranged vertically in a stacked configuration. The device may have a memory array that provides data, and also, the device may have an inter-layer data bus that vertically couples the memory array to the multi-layered logic structure. The inter-layer data bus may provide multiple data paths to the first logic layer and the second logic layer for reuse of the data provided by the memory array.

Multi-dimensional data path architecture

A system, an apparatus and methods for utilizing software and hardware portions of a neural network to fix, or hardwire, certain portions, while modifying other portions are provided. A first set of weights for layers of the first neural network are established, and selected weights are modified to generate a second set of weights, based on a second dataset. The second set of weights is then used to train a second neural network.

System, method and apparatus for training neural networks using multiple datasets

The present disclosure advantageously provides a mixed precision computation (MPC) unit for executing one or more mixed-precision layers of an artificial neural network (ANN). The MPC unit includes a multiplier circuit configured to input a pair of operands and output a product, a first adder circuit coupled to the multiplier circuit, a second adder circuit, coupled to the first adder circuit, configured to input a pair of operands, an accumulator circuit, coupled to the multiplier circuit and the first adder circuit, configured to output an accumulated value, and a controller, coupled to the multiplier circuit, the first adder circuit, the second adder circuit and the accumulator circuit, configured to input a mode control signal. The controller has a plurality of operating modes including a high precision mode, a low precision add mode and a low precision multiply mode.

Mixed-precision computation unit

The present disclosure advantageously provides an Optical Hardware Accelerator (OHA) for an Artificial Neural Network (ANN) that includes a communication bus interface, a memory, a controller, and an optical computing engine (OCE). The OCE is configured to execute an ANN model with ANN weights. Each ANN weight includes a quantized phase shift value &#x3b8;and a phase shift value &#x3d5;. The OCE includes a digital-to-optical (D/O) converter configured to generate input optical signals based on the input data, an optical neural network (ONN) configured to generate output optical signals based on the input optical signals, and an optical-to-digital (O/D) converter configured to generate the output data based on the output optical signals. The ONN includes a plurality of optical units (OUs), and each OU includes an optical multiply and accumulate (OMAC) module.

Artificial neural network optical hardware accelerator

The present disclosure advantageously provides a modulo operation unit that includes a first input configured to receive operand data, a second input configured to receive modulus data, an initial modulo stage, a sequence of intermediate modulo stages, and a final modulo stage.

Modulo operation unit

The present disclosure advantageously provides a pipelined accumulator that includes a data selector configured to receive a sequence of operands to be summed, an input register coupled to the data selector, an output register, coupled to the data selector, configured to store a sequence of partial sums and output a final sum, and a multi-stage add module coupled to the input register and the output register. The multi-stage add module is configured to store a sequence of partial sums and a final sum in a redundant format, and perform back-to-back accumulation into the output register.

Pipelined accumulator

A data processor receives a first set of processor instructions for combining a first matrix with a second matrix to produce a third matrix and generates a second set of processor instructions therefrom by identifying values of non-zero elements of the first matrix stored in a memory of the data processor and determining memory locations of elements of the second matrix. An instruction of the second set of processor instructions includes a determined memory location and/or an explicit value of an identified non-zero element. The second set of processor instructions is executed by the data processor. The second set of processor instructions may be generated by just-in-time compilation of the first set of processor instructions and may include instructions of a custom instruction set architecture.

Processor for sparse matrix computation

There is provided a data processing apparatus to perform an operation on a first matrix and a second matrix. The data processing apparatus includes receiver circuitry to receive elements of the first matrix, elements of the second matrix, and correspondence data to indicate where the elements of the first matrix are located in the first matrix. Determination circuitry performs, using the correspondence data, a determination of whether, for a given element of the first matrix in column i of the first matrix, a given element of the second matrix occurs in row i of the second matrix. Aggregation circuitry calculates an aggregation between a given row in the first matrix and a given column in the second matrix and includes: functional circuitry to perform, in dependence on the determination, a function on the given element of the first matrix and the given element of the second matrix to produce a partial result.

Apparatus and method for matrix operations

A circuit and method are provided for performing convolutional neural network computations for a neural network. The circuit includes a transposing buffer configured to receive actuation feature vectors along a first dimension and to output feature component vectors along a second dimension, a weight buffer configured to store kernel weight vectors along a first dimension and further configured to output kernel component vectors along a second dimension, and a systolic array configured to receive the kernel weight vectors along a first dimension and to receive the feature component vectors along a second dimension. The systolic array includes an array of multiply and accumulate (MAC) processing cells. Each processing cell is associated with an output value. The actuation feature vectors may be shifted into the transposing buffer along the first dimension and output feature component vectors may shifted out of the transposing buffer along the second dimension, providing efficient dataflow.

Systolic convolutional neural network

Growing community of inventors

Boylston, MA, United States of America

Matthew Mattina