Arm Limited

Apical Limited

Rune Holm

David Hennah Mansell

John Wakefield Brothers

Ganesh Suryanarayan Dasika

Jorn Nystad

Jørn Nystad

Aske Simon Christensen

Daren Croxford

Shidhartha Das

Andreas Due Engh-Halstvedt

Dominic Hugo Symes

Ian Michael Caulfield

Frode Heggelund

Stephen John Hill

Jayavarapu Srinivasa Rao

Jelena Milanovic

Timothy Holroyd Glauert

Suraj Sudhir

Michel Patrick Gabriel Emil Iwaniec

Edvard Feilding

Diego Lopez Recas

Elliott Maurice Simon Rosemarine

A computer-implemented method, performed in a neural processing system comprising control processor circuitry and arithmetic logic circuitry, of performing a convolution between an input feature map (IFM) and convolutional filter data, resulting in an output feature map (OFM). The method includes, obtaining in the control processor circuitry, dimensional characteristic parameters relating to dimensions of input work batch data arrays and positional characteristic parameters relating to positions of feature map content within the input work batches. The method also includes, in the arithmetic logic circuitry, performing convolutions between the input work batches, generated from the IFM based on the dimensional characteristic parameters and the positional characteristic parameters, and work batch filter data arrays corresponding to the filter to produce a plurality of output work batch data arrays. The plurality of output work batches are combined to generate an OFM.

Neural processing system

To perform neural network processing to modify an input data array to generate a corresponding output data array using a filter comprising an array of weight data, at least one of the input data array and the filter are subdivided into a plurality of portions, a plurality of neural network processing passes using the portions are performed, and the output generated by each processing pass is combined to provide the output data array.

Neural network processing

When performing a matrix-vector multiply operation for neural network processing, a set of one or more input vectors to be multiplied by a matrix of data values is scanned to identify data positions of the input vector(s) for which the data value is non-zero in at least one of the input vectors. For each of the data positions identified as having a non-zero value in at least one of the input vectors, the set of data values from the matrix of data values for that data position is fetched from memory and the matrix-vector multiply operation is performed using the data values for the input vectors for the data positions identified as being non-zero and the fetched set(s) of data values from the matrix of data values for those data position(s).

Performing matrix-vector multiply operations for neural networks on electronic devices

A method for performing neural network processing, and a corresponding data processing system. The data processing system is configured to define one or more tiles for use when reading a portion of an input feature map from memory or writing a portion of an output feature map to memory. The data processing system is also configured to provide information which allows positions falling within the defined one or more tiles to be mapped to memory locations to allow a processor to read data for an input feature map from memory or to write data for a portion of an output feature map to memory.

A neural network processor is disclosed that includes a combined convolution and pooling circuit that can perform both convolution and pooling operations. The circuit can perform a convolution operation by a multiply circuit determining products of corresponding input feature map and convolution kernel weight values, and an add circuit accumulating the products determined by the multiply circuit in storage. The circuit can perform an average pooling operation by the add circuit accumulating input feature map data values in the storage, a divisor circuit determining a divisor value, and a division circuit dividing the data value accumulated in the storage by the determined divisor value. The circuit can perform a maximum pooling operation by a maximum circuit determining a maximum value of input feature map data values, and storing the determined maximum value in the storage.

Neural net work processing

A single instruction multiple thread (SIMT) processor includes execution circuitry, prefetch circuitry and prefetch strategy selection circuitry. The prefetch strategy selection circuitry serves to detect one or more characteristics of a stream of program instructions that are being executed to identify whether or not a given data access instruction within a program will be executed a plurality of times. The prefetch strategy to use is selected from a plurality of selectable prefetch strategies in dependence upon the detection of such detected characteristics.

Prefetch strategy control for parallel execution of threads based on one or more characteristics of a stream of program instructions indicative that a data access instruction within a program is scheduled to be executed a plurality of times

Broadly speaking, embodiments of the present techniques provide a reconfigurable hardware-based artificial neural network, wherein weights for each neural network node of the artificial neural network are obtained via training performed external to the neural network.

Neural network including memory elements implemented at nodes

Techniques for performing matrix multiplication in a data processing apparatus are disclosed, comprising apparatuses, matrix multiply instructions, methods of operating the apparatuses, and virtual machine implementations. Registers, each register for storing at least four data elements, are referenced by a matrix multiply instruction and in response to the matrix multiply instruction a matrix multiply operation is carried out. First and second matrices of data elements are extracted from first and second source registers, and plural dot product operations, acting on respective rows of the first matrix and respective columns of the second matrix are performed to generate a square matrix of result data elements, which is applied to a destination register. A higher computation density for a given number of register operands is achieved with respect to vector-by-element techniques.

Register-based matrix multiplication with multiple matrices per register

A data processing system includes: a processor; a data interface for communication with a control unit, the processor being on one side of the data interface; internal storage accessible by the processor, the internal storage being on the same side of the data interface as the processor; and a register array accessible by the processor and comprising a plurality of registers, each register having a plurality of vector lanes. The storage is arranged to store control data indicating an ordered selection of vector lanes of one or more of the registers. The processor is arranged to, in response to receiving instruction data from a control unit, perform a swizzle operation in which data is selected from one or more source registers in the register array, and transferred to a destination register. The data is selected from vector lanes in accordance with control data stored in the internal storage.

Processor instruction specifying indexed storage region holding control data for swizzle operation

An apparatus and method are provided for executing a plurality of threads. The apparatus has processing circuitry arranged to execute the plurality of threads, with each thread executing a program to perform processing operations on thread data. Each thread has a thread identifier, and the thread data includes a value which is dependent on the thread identifier. Value generator circuitry is provided to perform a computation using the thread identifier of a chosen thread in order to generate the above mentioned value for that chosen thread, and to make that value available to the processing circuitry for use by the processing circuitry when executing the chosen thread. Such an arrangement can give rise to significant performance benefits when executing the plurality of threads on the apparatus.

Apparatus and method for executing a plurality of threads

A data processing apparatus () executes threads and includes a general program counter (PC) () identifying an instruction to be executed for at least a subset of the threads. Each thread has a thread PC (). The subset of threads has at least one lock parameter (-) for tracking exclusive access to shared resources. In response to a first instruction executed for a thread, the processor () modifies the at least one lock parameter (), (-) to indicate that the thread has gained exclusive access to the shared resource. In response to a second instruction, the processor modifies the at least one lock parameter (-) to indicate that the thread no longer has exclusive access. A selector () selects one of the subset of threads based on the at least one lock parameter (-) and sets the general PC () to the thread PC () of the selected thread.

Shared resources in a data processing apparatus for executing a plurality of threads

A data processing apparatushas processing circuitrywhich can process multiple parallel threads of processing. A shared instruction decoderdecodes program instructions to generate micro-operations to be processed by the processing circuitry. The instructions include at least one complex instruction which has multiple micro-operations. Multiple fetch unitsare provided for fetching the micro-operations generated by the decoderfor processing by the processing circuitry. Each fetch unitis associated with at least one of the threads. The decodergenerates the micro-operations of a complex instruction individually in response to separate decode requeststriggered by a fetch unit, each decode requestidentifying which micro-operation of the complex instruction is to be generated by the decoderin response to the decode request

Decoding a complex program instruction corresponding to multiple micro-operations

A data processing devicecomprises a plurality of storage circuits, which store a plurality of data elements of the bits in an interleaved manner. Data processing device also comprises a consumerwith a number of lanes. The consumer is able to individually access each of the plurality of storage circuitsin order to receive into the laneseither a subset of the plurality of data elements or y bits of each of the plurality of data elements. The consumeris also able to execute a common instruction of each of the plurality of lanes. The relationship of the bits is such that b is greater than y and is an integer multiple of y. Each of the plurality of storage circuitsstores at most y bits of each of the data elements. Furthermore, each of the storage circuitsstores at most y/b of the plurality of data elements. By carrying out the interleaving in this manner, the plurality of storage circuitscomprise no more than b/y storage circuits.

Data processing device and method for interleaved storage of data elements

A single instruction multiple thread (SIMT) processor includes scheduling circuitry for calculating a next scheduled execution point for execution circuits which execute respective threads corresponding to a common program. In addition to calculating the next scheduled execution point, the scheduling circuitry determines a runner up execution point which would have been determined as the next scheduled execution point if the threads which actually correspond to the next scheduled execution point had been removed from consideration. This runner up execution point is used to identify points of re-convergence within the program flow and as part of the operation of a static branch predictor.

Scheduling program instructions with a runner-up execution position

A data processing device includes processing circuitryfor executing a first memory access instruction to a first address of a memory deviceand a second memory access instruction to a second address of the memory device, the first address being different from the second address. The data processing device also includes prefetching circuitryfor prefetching data from the memory devicebased on a stride lengthand instruction analysis circuitryfor determining a difference between the first address and the second address. Stride refining circuitryis also provided to refine the stride length based on factors of the stride length and factors of the difference calculated by the instruction analysis circuitry

Growing community of inventors

Cambridge, United Kingdom

Rune Holm