Google Inc.

Groq, Inc.

Christopher Aaron Clark

Gregory Michael Thorson

Jonathan Alexander Ross

Andrew Everett Phelps

Dong Hyuk Woo

Olivier Temam

Thomas James Norrie

Norman Paul Jouppi

Dan Luu

William Lacy

Robert David Nuckolls

Chester Li

Travis Shamp

Laura L Manson

Matthew Leever Hedlund

Sushma Honnavara-Prasad

Vinayak Anand Gokhale

Jerry Sciarini

Pareesa Ameneh Golnari

Jeffrey Boll

A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

Vector processing unit

A circuit comprises an input register configured to receive an input vector of elements, a control register configured to receive a control vector of elements, wherein each element of the control vector corresponds to a respective element of the input vector, and wherein each element specifies a permutation of a corresponding element of the input vector, and a permute execution circuit configured to generate an output vector of elements corresponding to a permutation of the input vector. Generating each element of the output vector comprises accessing, at the input register, a particular element of the input vector, accessing, at the control register, a particular element of the control vector corresponding to the particular element of the input vector, and outputting the particular element of the input vector as an element at a particular position of the output vector that is selected based on the particular element of the control vector.

Permuting in a matrix-vector processor

A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.

Vector computation unit in a neural network processor

A circuit for transposing a matrix comprising reversal circuitry configured, for each of one or more diagonals of the matrix, to receive elements of the matrix in a first vector and generate a second vector that includes the elements of the matrix in an order that is a reverse of an order of the elements of the matrix in the first vector, and rotation circuitry configured, for each of the one or more diagonals of the matrix, to determine a number of positions by which to rotate the elements of the matrix in the second vector, receive the second vector of elements of the matrix, and generate a third vector that includes the elements of the matrix in the second vector in an order that is a rotation of the elements of the matrix in the second vector by the determined number of positions.

Transposing in a matrix-vector processor

In a system including vector registers storing right-hand side data and left-hand side data, first and second matrix staging registers, and a systolic array of processing cells for conducting matrix multiplication operations using the right-hand side data and left-hand side data, one or more processors load the right-hand side data from the vector registers to the first matrix staging register based on an instruction indicating whether to transpose the right-hand side data, load the left-hand side data from the vector registers into the second matrix staging register based on another instruction indicating whether to transpose the left-hand side data, load the right-hand side data from the first matrix staging register into the systolic array, and, in a cycle of the matrix multiplication operation, pass one or more columns of the left-hand side data from the second matrix staging register to a column of the systolic array.

Transposition of left-hand-side operand to systolic matrix multiplication

Embodiments of the present disclosure pertain to multimodal digital multiplier circuits and methods. In one embodiment, partial product outputs of digital multiplication circuits are selectively inverted based on a mode control signal. The mode control signal may be set based on a format of the operands input to the multiplier. Example embodiments of the disclosure may multiply combinations of signed and unsigned input operands using different modes.

Multimodal digital multiplication circuits and methods

In one embodiment, in a first mode, first and second input operands having a first data type are multiplied using one or more of a plurality of multipliers, and in second mode, a plurality of input operands having a second data type are multiplied using the plurality of multipliers. Accordingly, multiplier circuitry may process different input data types and share circuitry across the different modes. In some embodiments, in the first mode, products may be converted to a third data type, and in the second mode, multiple products may be concatenated. Values in the third data type, in the first mode, and concatenated values having the second data type, in the second mode, may be added across different multimodal multipliers to form a multiply-accumulator. In some embodiments, the plurality of multiply-accumulators may be configured in series.

Multiplier circuitry for multiplying operands of multiple data types

Growing community of inventors

Madison, WI, United States of America

Christopher Aaron Clark