Groq, Inc.

Google Inc.

Cray Incorporated

Intel Corporation

Leland Stanford Junior University

Silicon Graphics, Incorporated

Dennis Charles Abts

Steven Lee Scott

Jonathan Alexander Ross

Gregory Michael Thorson

John Thompson

William James Dally

Michael Roger Marty

Gregory J Hubbard

Robert L Alverson

Edwin Froese

Aaron Forest Godfrey

Van L Snyder

Peter Michael Klausler

Daniel Earle Gibson

Philip Michael Wells

John J Kim

Hong Liu

Abdul Kabbani

Robert Edman Felderman

Gerald A Schwoerer

Michael Higgins

Howard Pritchard

Andrew Ling

Roger A Bethard

Kelly J Marquardt

Tom Hawkins

Michael Bye

Michael Roberts

Garrin Kimmell

Matthew Boyd

A processor having a functional slice architecture is divided into a plurality of functional units ('tiles') organized into a plurality of slices. Each slice is configured to perform specific functions within the processor, which may include memory slices (MEM) for storing operand data, and arithmetic logic slices for performing operations on received operand data. The tiles of the processor are configured to stream operand data across a first dimension, and receive instructions across a second dimension orthogonal to the first dimension. The timing of data and instruction flows are configured such that corresponding data and instructions are received at each tile with a predetermined temporal relationship, allowing operand data to be transmitted between the slices of the processor without any accompanying metadata. Instead, each slice is able to determine what operations to perform on received data based upon the timing at which the data is received.

Memory design for a processor

Improved placement of memory and functional modules, 'tiles', within a tiled processor architecture are disclosed for linear algebra calculations involving vectors and matrices comprising large amounts of data. The improved placement places the data in close proximity to the functional modules performing calculations using the data. These modules enable these calculations to be performed more quickly while using less energy. These modules, in particular, improve the efficiency of the training and application of deep learning and artificial neural network systems. This Abstract and the independent Claims are concise signifiers of embodiments of the claimed inventions. The Abstract does not limit the scope of the claimed inventions.

Streaming processor architecture

A system contains a network of processors arranged in a plurality of nodes. Each node comprises a respective plurality of processors connected via local links, and different nodes are connected via global links. The processors of the network communicate with each other to establish a global counter for the network, enabling deterministic communication between the processors of the network. A compiler is configured to explicitly schedule communication traffic across the global and local links of the network of processors based upon the deterministic links between the processors, which enable software-scheduled networking with explicit send or receive instructions executed by functional units of the processors at specific times, to establish a specific ordering of operations performed by the network of processors. In some embodiments, the processors of the network of processors are tensor streaming processors (TSPs).

Software-defined tensor streaming multiprocessor for large-scale machine learning

Embodiments are directed to a processor having a functional slice architecture. The processor is divided into tiles (or functional units) organized into a plurality of functional slices. The functional slices are configured to perform specific operations within the processor, which includes memory slices for storing operand data and arithmetic logic slices for performing operations on received operand data (e.g., vector processing, matrix manipulation). The processor includes a plurality of functional slices of a module type, each functional slice having a plurality of tiles. The processor further includes a plurality of data transport lanes for transporting data in a direction indicated in a corresponding instruction. The processor also includes a plurality of instruction queues, each instruction queue associated with a corresponding functional slice of the plurality of functional slices, wherein the instructions in the instruction queues comprise a functional slice specific operation code.

Instruction format and instruction set architecture for tensor streaming processor

Compiler operations for tensor streaming processor

Embodiments are directed to a deterministic streaming system with one or more deterministic streaming processors each having an array of processing elements and a first deterministic memory coupled to the processing elements. The deterministic streaming system further includes a second deterministic memory with multiple data banks having a global memory address space, and a controller. The controller initiates retrieval of first data from the data banks of the second deterministic memory as a first plurality of streams, each stream of the first plurality of streams streaming toward a respective group of processing elements of the array of processing elements. The controller further initiates writing of second data to the data banks of the second deterministic memory as a second plurality of streams, each stream of the second plurality of streams streaming from the respective group of processing elements toward a respective data bank of the second deterministic memory.

Deterministic memory for tensor streaming processors

A computational array is implemented in which all operands and results are loaded or output from a single side of the array. The computational array comprises a plurality of cells arranged in n rows and m columns, each configured to produce a processed value based upon a weight value and an activation value. The cells receive weight and activation values via colinear weight and activation transmission channels that each extend across a first side edge of the computational array to provide weight values and activation values to the cells of the array. In addition, result values produced at a top cell of each of the m columns of the array are routed through the array to be output from the same first side edge of the array at a same relative timing at which the result values were produced.

Loading operands and outputting results from a multi-dimensional array using only a single side

Tensor streaming processor architecture

Processor architecture

A predictive model utilizes a set of coefficients for processing received input data. To reduce memory usage storing the coefficients, a compression circuit compresses the set of coefficients prior to storage by generating a cumulative count distribution of the coefficient values, and identifying a distribution function approximating the cumulative count distribution. Function parameters for the determined function are stored in a memory and used by a decompression circuit to apply the function the compressed coefficients to determine the decompressed component values. Storing the function parameters may consume less memory in comparison to storing a look-up table for decompression, and may reduce an amount of memory look-ups required during decompression.

Decompression of model parameters using functions based upon cumulative count distributions

A multiprocessor computer system comprises a first node operable to access memory local to a remote node by receiving a virtual memory address from a requesting entity in node logic in the first node. The first node creates a network address from the virtual address received in the node logic, where the network address is in a larger address space than the virtual memory address, and sends a fast memory access request from the first node to a network node identified in the network address.

Extended fast memory access in a multiprocessor computer system

Growing community of inventors

Eleva, WI, United States of America

Dennis Charles Abts