Arm Limited

Paul Nicholas Whatmough

Matthew Mattina

Shidhartha Das

David Michael Bull

Zhi-Gang Liu

Supreet Jeloka

Jesse Garrett Beu

Mudit Bhargava

Zhigang Liu

Dibakar Gope

Yuhao Zhu

David Hennah Mansell

Glen Arnold Rosendale

Ian Rudolf Bratt

Daniel Kershaw

Saurabh Pijuskumar Sinha

Fernando Garcia Redondo

Reiley Jeyapaul

George Christopher Smart

Jeremy Constantin

Chuteng Zhou

Patrick Thomas Hansen

Igor Fedorov

Matthew David Haddon

Teyuh Alice Chou

A compute-in-memory (CIM) array module and a method for performing dynamic saturation detection for a CIM array are provided. The CIM array module includes a CIM array, saturation detection units (SDUs) and a controller. The CIM array includes selectable row signal lines, column signal lines and cells. Each cell is located at an intersection of a selectable row signal line and a column signal line, and each cell has a programmable conductance. The SDUs are selectively coupled to at least one column signal line, and each SDU is configured to, for each column signal line, generate an analog signal, and identify the column signal line as a saturated column signal line when a voltage of the analog signal is greater than a saturation threshold voltage, or a current of the analog signal is greater than a saturation threshold current.

Memory for artificial neural network accelerator

Methods and systems for detecting errors when performing a convolutional operation is provided. Predicted checksum data, corresponding to input checksum data and kernel checksum data, is obtained. The convolutional operation is performed to obtain an output feature map. Output checksum data is generated and the predicted checksum data and the output checksum data are compared, the comparing taking account of partial predicted checksum data configured to correct for a lack of padding when performing the convolution operation, wherein the partial predicted checksum data corresponds to input checksum data for a subset of the values in the input feature map and kernel checksum data for a subset of the values in the kernel.

Error detection in convolutional operations

An artificial neural network (ANN) accelerator is provided. The ANN accelerator includes digital controlled oscillators (DCOs), digital-to-time converters (DTCs) and a mixed-signal multiply-and-accumulate (MAC) array. Each DCO generates a first analog operand signal based on a first digital data value, and transmits the first analog operand signal along a respective column signal line. Each DTC generates a second analog operand signal based on a second digital data value, and transmits the second analog operand signal along a respective row signal line. The mixed-signal MAC array is coupled to the row and column signal lines, and includes mixed-signal MAC units. Each mixed-signal MAC unit includes an integrated clock gate (ICG) that generates a digital product signal based on the first and second analog operand signals, and a counter circuit that increments or decrements a count value stored in a register based on the digital product signal.

Mixed-signal artificial neural network accelerator

A memory for an artificial neural network (ANN) accelerator is provided. The memory includes a first bank, a second bank and a bank selector. Each bank includes at least two word lines and a plurality of write word selectors. Each word line stores a plurality of words, and each word has a plurality of bytes. Each write word selector has an input port and a plurality of output ports, is coupled to a corresponding word in each word line, and is configured to select a byte of the corresponding word of a selected word line based on a byte select signal. The bank selector is coupled to the write word selectors of the first bank and the second bank, and configured to select a combination of write word selectors from at least one of the first bank and the second bank based on a bank select signal.

Memory for an artificial neural network accelerator

The present disclosure advantageously provides a system including a memory, a processor, and a circuitry to execute one or more mixed precision layers of an artificial neural network (ANN), each mixed precision layer including high-precision weight filters and low precision weight filters. The circuitry is configured to perform one or more calculations on an input feature map having a plurality of input channels (c) using the high precision weight filters to create a high precision output feature map having a first number of output channels (k), perform one or more calculations on the input feature map using the low precision weight filters to create a low precision output feature map having a second number of output channels (c−k), and concatenate the high precision output feature map and the low precision output feature map to create a unified output feature map having a plurality of output channels (c).

Hybrid filter banks for artificial neural networks

A system and method for multiplying matrices are provided. The system includes a processor coupled to a memory and a matrix multiply accelerator (MMA) coupled to the processor. The MMA is configured to multiply, based on a bitmap, a compressed first matrix and a second matrix to generate an output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the compressed first matrix and a jcolumn of the second matrix based on the bitmap. Or, the MMA is configured to multiply, based on the bitmap, the second matrix and the compressed first matrix and to generate the output matrix including, for each element i,j of the output matrix, a calculation of a dot product of an irow of the second matrix and a jcolumn of the compressed first matrix based on the bitmap.

Time domain unrolling sparse matrix multiplication system and method

A non-volatile memory (NVM) crossbar for an artificial neural network (ANN) accelerator is provided. The NVM crossbar includes row signal lines configured to receive input analog voltage signals, multiply-and-accumulate (MAC) column signal lines, a correction column signal line, a MAC cell disposed at each row signal line and MAC column signal line intersection, and a correction cell disposed at each row signal line and correction column signal line intersection. Each MAC cell includes one or more programmable NVM elements programmed to an ANN unipolar weight, and each correction cell includes one or more programmable NVM elements. Each MAC column signal line generates a MAC signal based on the input analog voltage signals and the respective MAC cells, and the correction column signal line generates a correction signal based on the input analog voltage signals and the correction cells. Each MAC signal is corrected based on the correction signal.

Non-volatile memory accelerator for artificial neural networks

A method for processing video data, comprising: receiving raw video data, representative of a plurality of frames; detecting, using the raw video data, one or more regions of interest in a detection frame that belongs to the plurality of frames, for example using a region proposal network; performing a cropping process on a portion of the raw video data representative of the detection frame, based on the regions of interest, so as to generate cropped raw video data; performing image processing on the cropped raw video data, including demosaicing, so as to generate processed image data for the detection frame; and analyzing the processed image data, for example using an object detection process, to determine information relating to at least one of said one or more regions of interest.

Video data processing

The present disclosure advantageously provides a matrix expansion unit that includes an input data selector, a first register set, a second register set, and an output data selector. The input data selector is configured to receive first matrix data in a columnwise format. The first register set is coupled to the input data selector, and includes a plurality of data selectors and a plurality of registers arranged in a first shift loop. The second register set is coupled to the data selector, and includes a plurality of data selectors and a plurality of registers arranged in a second shift loop. The output data selector is coupled to the first register set and the second register set, and is configured to output second matrix data in a rowwise format.

Hardware accelerator for IM2COL operation

Various implementations described herein are directed to a device having a multi-layered logic structure with a first logic layer and a second logic layer arranged vertically in a stacked configuration. The device may have a memory array that provides data, and also, the device may have an inter-layer data bus that vertically couples the memory array to the multi-layered logic structure. The inter-layer data bus may provide multiple data paths to the first logic layer and the second logic layer for reuse of the data provided by the memory array.

Multi-dimensional data path architecture

A system, an apparatus and methods for utilizing software and hardware portions of a neural network to fix, or hardwire, certain portions, while modifying other portions are provided. A first set of weights for layers of the first neural network are established, and selected weights are modified to generate a second set of weights, based on a second dataset. The second set of weights is then used to train a second neural network.

System, method and apparatus for training neural networks using multiple datasets

The present disclosure advantageously provides a hardware accelerator for an artificial neural network (ANN), including a communication bus interface, a memory, a controller, and at least one processing engine (PE). The communication bus interface is configured to receive a plurality of finetuned weights associated with the ANN, receive input data, and transmit output data. The memory is configured to store the plurality of finetuned weights, the input data and the output data. The PE is configured to receive the input data, execute an ANN model using a plurality of fixed weights associated with the ANN and the plurality of finetuned weights, and generate the output data. Each finetuned weight corresponds to a fixed weight.

Sparse finetuning for artificial neural networks

The present disclosure advantageously provides a mixed precision computation (MPC) unit for executing one or more mixed-precision layers of an artificial neural network (ANN). The MPC unit includes a multiplier circuit configured to input a pair of operands and output a product, a first adder circuit coupled to the multiplier circuit, a second adder circuit, coupled to the first adder circuit, configured to input a pair of operands, an accumulator circuit, coupled to the multiplier circuit and the first adder circuit, configured to output an accumulated value, and a controller, coupled to the multiplier circuit, the first adder circuit, the second adder circuit and the accumulator circuit, configured to input a mode control signal. The controller has a plurality of operating modes including a high precision mode, a low precision add mode and a low precision multiply mode.

Mixed-precision computation unit

A memory for an artificial neural network (ANN) accelerator is provided. The memory includes a first bank, a second bank and a bank selector. Each bank includes at least two word lines and a plurality of read word selectors. Each word line stores a plurality of words, and each word has a plurality of bytes. Each read word selector has a plurality of input ports and an output port, is coupled to a corresponding word in each word line, and is configured to select a byte of the corresponding word of a selected word line based on a byte select signal. The bank selector is coupled to the read word selectors of the first bank and the second bank, and configured to select a combination of read word selectors from at least one of the first bank and the second bank based on a bank select signal.

The present disclosure advantageously provides a pipelined accumulator that includes a data selector configured to receive a sequence of operands to be summed, an input register coupled to the data selector, an output register, coupled to the data selector, configured to store a sequence of partial sums and output a final sum, and a multi-stage add module coupled to the input register and the output register. The multi-stage add module is configured to store a sequence of partial sums and a final sum in a redundant format, and perform back-to-back accumulation into the output register.

Growing community of inventors

Cambridge, MA, United States of America

Paul Nicholas Whatmough