Arm Limited

Emc Corporation

Samsung Electronics Co., Ltd.

Zhi-Gang Liu

Paul Nicholas Whatmough

Matthew Mattina

Steven T McClure

Supreet Jeloka

Mudit Bhargava

Alesia Tringale

Sehwan Lee

Saurabh Pijuskumar Sinha

Robert DeCrescenzo

Jerome Cartmell

Reiley Jeyapaul

Scott B Gordon

Timothy Johnson

Matthew David Haddon

Jun Woo Jang

Dongkyun Kim

John Fremont Brown, Iii

Igor Fedorov

A system and method for multiplying matrices, and method for training a convolutional neural network (CNN), are provided. The system includes a processor and a matrix multiply accelerator (MMA). The processor is configured to generate, based on an input tensor, a number of basic block matrices, each basic block matrix including a number of elements; for each basic block matrix: prune, based on a sparsity value, the elements of the basic block matrix, generate a mask for the basic block matrix, each mask including a number of bits, each bit corresponding to a different element of the basic block matrix, and compress the basic block matrix to generate a compressed basic block matrix having fewer elements than the basic block matrix. The MMA is configured to multiply, based on the masks, the compressed basic block matrices and a weight matrix to generate an output matrix.

Activation compression method for deep learning acceleration

An integrated circuit includes: a central processing unit (CPU) core; an accelerator; and an acceleration instruction queue connected to the CPU core and the accelerator. The CPU core is to: fetch and decode one or more instructions from among an instruction sequence in a programmed order; determine an instruction from among the one or more instructions containing an acceleration workload encoded therein; and queue the instruction containing the acceleration workload encoded therein in the acceleration instruction queue.

CPU tight-coupled accelerator

Methods and systems for detecting errors when performing a convolutional operation is provided. Predicted checksum data, corresponding to input checksum data and kernel checksum data, is obtained. The convolutional operation is performed to obtain an output feature map. Output checksum data is generated and the predicted checksum data and the output checksum data are compared, the comparing taking account of partial predicted checksum data configured to correct for a lack of padding when performing the convolution operation, wherein the partial predicted checksum data corresponds to input checksum data for a subset of the values in the input feature map and kernel checksum data for a subset of the values in the kernel.

Error detection in convolutional operations

A memory for an artificial neural network (ANN) accelerator is provided. The memory includes a first bank, a second bank and a bank selector. Each bank includes at least two word lines and a plurality of write word selectors. Each word line stores a plurality of words, and each word has a plurality of bytes. Each write word selector has an input port and a plurality of output ports, is coupled to a corresponding word in each word line, and is configured to select a byte of the corresponding word of a selected word line based on a byte select signal. The bank selector is coupled to the write word selectors of the first bank and the second bank, and configured to select a combination of write word selectors from at least one of the first bank and the second bank based on a bank select signal.

Memory for an artificial neural network accelerator

A system and method for multiplying matrices are provided. The system includes a processor coupled to a memory and a matrix multiply accelerator (MMA) coupled to the processor. The MMA is configured to multiply, based on a bitmap, a compressed first matrix and a second matrix to generate an output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the compressed first matrix and a jcolumn of the second matrix based on the bitmap. Or, the MMA is configured to multiply, based on the bitmap, the second matrix and the compressed first matrix and to generate the output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the second matrix and a jcolumn of the compressed first matrix based on the bitmap.

Time domain unrolling sparse matrix multiplication system and method

The present disclosure advantageously provides a matrix expansion unit that includes an input data selector, a first register set, a second register set, and an output data selector. The input data selector is configured to receive first matrix data in a columnwise format. The first register set is coupled to the input data selector, and includes a plurality of data selectors and a plurality of registers arranged in a first shift loop. The second register set is coupled to the data selector, and includes a plurality of data selectors and a plurality of registers arranged in a second shift loop. The output data selector is coupled to the first register set and the second register set, and is configured to output second matrix data in a rowwise format.

Hardware accelerator for IM2COL operation

Various implementations described herein are directed to a device having a multi-layered logic structure with a first logic layer and a second logic layer arranged vertically in a stacked configuration. The device may have a memory array that provides data, and also, the device may have an inter-layer data bus that vertically couples the memory array to the multi-layered logic structure. The inter-layer data bus may provide multiple data paths to the first logic layer and the second logic layer for reuse of the data provided by the memory array.

Multi-dimensional data path architecture

The present disclosure advantageously provides an Optical Hardware Accelerator (OHA) for an Artificial Neural Network (ANN) that includes a communication bus interface, a memory, a controller, and an optical computing engine (OCE). The OCE is configured to execute an ANN model with ANN weights. Each ANN weight includes a quantized phase shift value &#x3b8;and a phase shift value &#x3d5;. The OCE includes a digital-to-optical (D/O) converter configured to generate input optical signals based on the input data, an optical neural network (ONN) configured to generate output optical signals based on the input optical signals, and an optical-to-digital (O/D) converter configured to generate the output data based on the output optical signals. The ONN includes a plurality of optical units (OUs), and each OU includes an optical multiply and accumulate (OMAC) module.

Artificial neural network optical hardware accelerator

A memory for an artificial neural network (ANN) accelerator is provided. The memory includes a first bank, a second bank and a bank selector. Each bank includes at least two word lines and a plurality of read word selectors. Each word line stores a plurality of words, and each word has a plurality of bytes. Each read word selector has a plurality of input ports and an output port, is coupled to a corresponding word in each word line, and is configured to select a byte of the corresponding word of a selected word line based on a byte select signal. The bank selector is coupled to the read word selectors of the first bank and the second bank, and configured to select a combination of read word selectors from at least one of the first bank and the second bank based on a bank select signal.

The present disclosure advantageously provides a modulo operation unit that includes a first input configured to receive operand data, a second input configured to receive modulus data, an initial modulo stage, a sequence of intermediate modulo stages, and a final modulo stage.

Modulo operation unit

The present disclosure advantageously provides a pipelined accumulator that includes a data selector configured to receive a sequence of operands to be summed, an input register coupled to the data selector, an output register, coupled to the data selector, configured to store a sequence of partial sums and output a final sum, and a multi-stage add module coupled to the input register and the output register. The multi-stage add module is configured to store a sequence of partial sums and a final sum in a redundant format, and perform back-to-back accumulation into the output register.

Pipelined accumulator

The present disclosure advantageously provides a system, matrix multiply accelerator (MMA) and method for efficiently multiplying matrices. The MMA includes a vector register to store the row vectors of one input matrix, a vector register to store the column vectors of another input matrix, a vector register to store an output matrix, and an array of vector multiply and accumulate (VMAC) units coupled to the vector registers. Each VMAC unit is coupled to at least two row vector signal lines and at least two column vector signal lines, and is configured to calculate the dot product for one element i,j of the output matrix by multiplying each row vector formed from the irow of the first matrix with a corresponding column vector formed from the jcolumn of the second matrix to generate intermediate products, and accumulate the intermediate products into a scalar value.

Matrix multiplication system, apparatus and method

The present disclosure advantageously provides a system method for efficiently multiplying matrices with elements that have a value of 0. A bitmap is generated for each matrix. Each bitmap includes a bit position for each matrix element. The value of each bit is set to 0 when the value of the corresponding matrix element is 0, and to 1 when the value of the corresponding matrix element is not 0. Each matrix is compressed into a compressed matrix, which will have fewer elements with a value of 0 than the original matrix. Each bitmap is then adjusted based on the corresponding compressed matrix. The compressed matrices are then multiplied to generate an output matrix. For each element i,j in the output matrix, a dot product of the irow of the first compressed matrix and the jcolumn of the second compressed matrix is calculated based on the bitmaps.

Matrix multiplication system and method

Described are techniques for processing a request to access global memory. For a first processor included on a first of a plurality of boards connected by a fabric, a logical address is determined for a global memory location in a system global memory. A first physical address for the logical address is determined. It is determined whether the first physical address is included in a first global partition of the first board. If so, first processing is performed including updating a memory map to map a window of the first processor's logical address space to a physical memory segment located within the first global partition. Otherwise, if the first physical address is included in a second of the plurality of global partitions physically located on one of the plurality of boards other than said first board, second processing is performed to issue the request over the fabric.

Techniques for global memory management and request processing

Detecting data errors in connection with a data transfer process includes performing an XOR operation on a plurality of data blocks to obtain a data block XOR result. An XOR operation may be performed on a plurality of cyclic redundancy check (CRC) codes associated with the plurality of data blocks to obtain a CRC XOR result. The data block XOR result and the CRC XOR result may be used to determine whether an error exists in the plurality of data blocks. The system may be used in connection with local IO transfers and in connection with local CPU XOR operations for a RAID system in which data may be mirrored, striped or otherwise distributed across multiple storage devices.

Growing community of inventors

Westford, MA, United States of America

Zhi-Gang Liu