Arm Limited

Jesse Garrett Beu

Matthew Mattina

Paul Nicholas Whatmough

Joshua Randall

Mbou Eyole

Alejandro Rico Carro

Alejandro Martinez Vicente

Timothy Hayes

Dibakar Gope

Jamshed Jalal

Christopher Neal Hinds

Lee Evan Eisen

Michael Filippo

Tushar P Ringe

Ganesh Suryanarayan Dasika

Jose Alberto Joao

Zhigang Liu

Urmish Ajit Thakker

Jin Tao

The present disclosure advantageously provides a system including a memory, a processor, and a circuitry to execute one or more mixed precision layers of an artificial neural network (ANN), each mixed precision layer including high-precision weight filters and low precision weight filters. The circuitry is configured to perform one or more calculations on an input feature map having a plurality of input channels (c) using the high precision weight filters to create a high precision output feature map having a first number of output channels (k), perform one or more calculations on the input feature map using the low precision weight filters to create a low precision output feature map having a second number of output channels (c−k), and concatenate the high precision output feature map and the low precision output feature map to create a unified output feature map having a plurality of output channels (c).

Hybrid filter banks for artificial neural networks

An apparatus and method are provided for receiving a request from a plurality of processing units, where multiple of those processing units have associated cache storage. A snoop unit is used to implement a cache coherency protocol when a request is received that identifies a cacheable memory address. The snoop unit has snoop filter storage comprising a plurality of snoop filter tables organized in a hierarchical arrangement. The snoop filter tables comprise a primary snoop filter table at a highest level in the hierarchy, and each snoop filter table at a lower level in the hierarchy forms a backup snoop filter table for an adjacent snoop filter table at a higher level in the hierarchy. Each snoop filter table is arranged as a multi-way set associative storage structure, and each backup snoop filter table has a different number of sets than are provided in the adjacent snoop filter table.

Apparatus and method for providing coherence data for use when implementing a cache coherency protocol

The present disclosure advantageously provides a system and a method for skipping recurrent neural network (RNN) state updates using a skip predictor. Sequential input data are received and divided into sequences of input data values, each input data value being associated with a different time step for a pre-trained RNN model. At each time step, the hidden state vector for a prior time step is received from the pre-trained RNN model, and a determination, based on the input data value and the hidden state vector for at least one prior time step, is made whether to provide or not provide the input data value associated with the time step to the pre-trained RNN model for processing. When the input data value is not provided, the pre-trained RNN model does not update its hidden state vector. Importantly, the skip predictor is trained without retraining the pre-trained RNN model.

Skip predictor for pre-trained recurrent neural networks

A system, an apparatus and methods for utilizing software and hardware portions of a neural network to fix, or hardwire, certain portions, while modifying other portions are provided. A first set of weights for layers of the first neural network are established, and selected weights are modified to generate a second set of weights, based on a second dataset. The second set of weights is then used to train a second neural network.

System, method and apparatus for training neural networks using multiple datasets

An apparatus comprises snoop filter storage circuitry to store snoop filter entries corresponding to addresses and comprising sharer information. Control circuitry selects which sharers, among a plurality of sharers capable of holding cached data, should be issued with snoop requests corresponding to a target address, based on the sharer information of the snoop filter entry corresponding to the target address. The control circuitry is capable of setting a given snoop filter entry corresponding to a given address to an imprecise encoding in which the sharer information provides an imprecise description of which sharers hold cached data corresponding to the given address, and the given snoop filter entry comprises at least one sharer count value indicative of a number of sharers holding cached data corresponding to the given address.

Snoop filter with imprecise encoding

The present disclosure advantageously provides a mixed precision computation (MPC) unit for executing one or more mixed-precision layers of an artificial neural network (ANN). The MPC unit includes a multiplier circuit configured to input a pair of operands and output a product, a first adder circuit coupled to the multiplier circuit, a second adder circuit, coupled to the first adder circuit, configured to input a pair of operands, an accumulator circuit, coupled to the multiplier circuit and the first adder circuit, configured to output an accumulated value, and a controller, coupled to the multiplier circuit, the first adder circuit, the second adder circuit and the accumulator circuit, configured to input a mode control signal. The controller has a plurality of operating modes including a high precision mode, a low precision add mode and a low precision multiply mode.

Mixed-precision computation unit

A data processor receives a first set of processor instructions for combining a first matrix with a second matrix to produce a third matrix and generates a second set of processor instructions therefrom by identifying values of non-zero elements of the first matrix stored in a memory of the data processor and determining memory locations of elements of the second matrix. An instruction of the second set of processor instructions includes a determined memory location and/or an explicit value of an identified non-zero element. The second set of processor instructions is executed by the data processor. The second set of processor instructions may be generated by just-in-time compilation of the first set of processor instructions and may include instructions of a custom instruction set architecture.

Processor for sparse matrix computation

Non-volatile storage circuitry is provided as primary storage accessible to processing circuitry, e.g. as registers, a cache, scratchpad memory, TLB or on-chip RAM. Power control circuitry powers down a given region of the non-volatile storage circuitry when information stored in said given region is not being used. This provides opportunities for more frequent power savings than would be possible if primary storage was implemented using volatile storage.

Non-volatile storage circuitry accessible as primary storage for processing circuitry

An apparatus is provided for receiving requests from a plurality of processing units, at least some of which may have associated cache storage. A snoop unit implements a cache coherency protocol when a request received by the apparatus identifies a cacheable memory address. Snoop filter storage is provided comprising an N-way set associative storage structure with a plurality of entries. Each entry stores coherence data for an associated address range identifying a memory block, and the coherence data is used to determine which cache storages need to be subjected to a snoop operation when implementing the cache coherency protocol in response to a received request. The snoop filter storage stores coherence data for memory blocks of at least a plurality P of different size granularities, and is organised as a plurality of at least P banks that are accessible in parallel, where each bank has entries within each of the N-ways of the snoop filter storage. The snoop control circuitry controls access to the snoop filter storage, and is responsive to a received address to create a group of indexes, the group of indexes comprising an index for each different size granularity amongst the P different size granularities, and each index in the group being constrained so as to identify an entry in a different bank of the snoop filter storage. The snoop control circuitry uses the group of indexes to perform a lookup operation in parallel within the snoop filter storage in order to determine, taking into account each of the different size granularities, whether an entry stores coherence data for the received address.

Apparatus and method for maintaining cache coherence data for memory blocks of different size granularities using a snoop filter storage comprising an n-way set associative storage structure

An apparatus and method of operating the apparatus are provided for performing a count operation. Instruction decoder circuitry is responsive to a count instruction specifying an input data item to generate control signals to control the data processing circuitry to perform a count operation. The count operation determines a count value indicative of a number of input elements of a subset of elements in the specified input data item which have a value which matches a reference value in a reference element in a reference data item. A plurality of count operations may be performed to determine a count data item corresponding to the input data item. A register scatter storage instruction, a gather index generation instruction, and respective apparatuses responsive to them, as well as simulator implementations, are also provided.

Counting elements in data items in a data processing apparatus

An apparatus and a method of operating the apparatus are provided for performing a comparison operation to match a given sequence of values within an input vector. Instruction decoder circuitry is responsive to a string match instruction specifying a segment of an input vector to generate control signals to control the data processing circuitry to perform a comparison operation. The comparison operation determines a comparison value indicative of whether each input element of a required set of consecutive input elements of the segment has a value which matches a respective value in consecutive reference elements of the reference data item. A plurality of comparison operations may be performed to determine a match vector corresponding to the segment of the input vector to indicate the start position of the substring in the input vector. A string match instruction, as well as simulator virtual machine implementations, are also provided.

Matching consecutive values in a data processing apparatus

An apparatus and method of operating a data processing apparatus are provided. The data processing circuitry is responsive to a hint instruction to then assert at least one performance modifying control signal, when subsequently generating control signals for other data processing instructions. This causes the data processing functional hardware which performs the data processing operations defined by the data processing instructions to operate in a modified manner, although the data processing results produced do not change in dependence on whether the at least one performance modifying control signal is asserted.

Growing community of inventors

Austin, TX, United States of America

Jesse Garrett Beu