Google Inc.

Matthew Leever Hedlund

Andrew Everett Phelps

Thomas James Norrie

Norman Paul Jouppi

Gurushankar Rajamani

Vinayak Anand Gokhale

Christopher Aaron Clark

Matthew William Ashcraft

Sushma Honnavara-Prasad

Indranil Chakraborty

Pareesa Ameneh Golnari

Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes a first memory, respective first and second processor cores, and a shared memory. The first memory provides data for performing computations to generate an output for a neural network layer. Each of the first and second cores include a vector memory for storing vector values derived from the data provided by the first memory. The shared memory is disposed generally intermediate the first memory and at least one core and includes: i) a direct memory access (DMA) data path configured to route data between the shared memory and the respective vector memories of the first and second cores and ii) a load-store data path configured to route data between the shared memory and respective vector registers of the first and second cores.

Shared scratchpad memory with parallel load-store

A system including one or more processors configured to receive a transpose instruction indicating to transpose a source matrix to a result matrix, provide data elements of the source matrix to input switching circuits, reorder the data elements using the input switching circuits, provide the data elements from the input switching circuits to one or more lanes of a datapath, provide the data elements from the datapath to output switching circuits, undo the reordering of the data elements using the output switching circuits, and provide the data elements from the output switching circuits to a result matrix. Each respective lane of the datapath receiving data elements receives multiple data elements directed to different respective non-overlapping portions of the lane.

Transposing at-speed in a vector-matrix accelerator

In a system including vector registers storing right-hand side data and left-hand side data, first and second matrix staging registers, and a systolic array of processing cells for conducting matrix multiplication operations using the right-hand side data and left-hand side data, one or more processors load the right-hand side data from the vector registers to the first matrix staging register based on an instruction indicating whether to transpose the right-hand side data, load the left-hand side data from the vector registers into the second matrix staging register based on another instruction indicating whether to transpose the left-hand side data, load the right-hand side data from the first matrix staging register into the systolic array, and, in a cycle of the matrix multiplication operation, pass one or more columns of the left-hand side data from the second matrix staging register to a column of the systolic array.

Transposition of left-hand-side operand to systolic matrix multiplication

Methods, systems, and apparatus, including computer-readable media, are described for performing vector reductions using a shared scratchpad memory of a hardware circuit having processor cores that communicate with the shared memory. For each of the processor cores, a respective vector of values is generated based on computations performed at the processor core. The shared memory receives the respective vectors of values from respective resources of the processor cores using a direct memory access (DMA) data path of the shared memory. The shared memory performs an accumulation operation on the respective vectors of values using an operator unit coupled to the shared memory. The operator unit is configured to accumulate values based on arithmetic operations encoded at the operator unit. A result vector is generated based on performing the accumulation operation using the respective vectors of values.

Vector reductions using shared scratchpad memory

Growing community of inventors

Madison, WI, United States of America

Matthew Leever Hedlund