Google Inc.

Enfabrica Corporation

Thomas James Norrie

Andrew Everett Phelps

Norman Paul Jouppi

Gregory Michael Thorson

Naveen Kumar

Christopher Aaron Clark

Jonathan Alexander Ross

Reginald Clifford Young

Shrijeet Mukherjee

Dan Luu

William Lacy

Matthew Leever Hedlund

Albert Borchers

Andrew Swing

Rahul Nagarajan

Christopher Daniel Leary

Oliver Edward Bowen

Benjamin Steel Gelb

Matthew William Ashcraft

Thejasvi Magudilu Vijayaraj

Mark William Gottscho

Gurushankar Rajamani

Houle Gan

Shimon Muller

Rochan Sankar

Frederic Vecoven

Kiran Seshadri

Chetan Loke

Olivier Temam

Shuai Jiang

Robert S Sprinkle

Gregory Sizikov

Georgios K Konstadinidis

Hema Hariharan

Jason W Klaus

Ariel Hendel

Arpith Chacko Jacob

Suvinay Subramanian

Gurjeet Singh

Robert Ashby Armistead, Iii

Binayak Roy

Sushma Honnavara-Prasad

Vinayak Anand Gokhale

John Greth

Pareesa Ameneh Golnari

Jonathan Mayer

Andrews T Swing

Oliver Temam

Albert T Borchers

Andrew T Swing

Aspects of the disclosure are directed to methods, systems, and apparatuses using an instruction prefetch pipeline architecture that provides good performance without the complexity of a full cache coherent solution deployed in conventional CPUs. The architecture can include components which can be used to construct an instruction prefetch pipeline, including instruction memory (TiMem), instruction buffer (iBuf), a prefetch unit, and an instruction router.

Cooperative instruction prefetch on multicore system

A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers is disclosed. In some embodiments, a switch includes a plurality of hosts associated with a plurality of hierarchies, one or more endpoints associated with one or more of the plurality of hierarchies, and a switch communicatively connectable to the plurality of hosts and the one or more endpoints. The switch is configured to: receive a transaction layer packet (TLP); determine a policy group identifier based on parsing and processing the TLP; perform packet forward matching based on the policy group identifier and destination fields of the TLP; based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and forward the TLP to an appropriate destination link.

System and method for ghost bridging

A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

Vector processing unit

Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes a first memory, respective first and second processor cores, and a shared memory. The first memory provides data for performing computations to generate an output for a neural network layer. Each of the first and second cores include a vector memory for storing vector values derived from the data provided by the first memory. The shared memory is disposed generally intermediate the first memory and at least one core and includes: i) a direct memory access (DMA) data path configured to route data between the shared memory and the respective vector memories of the first and second cores and ii) a load-store data path configured to route data between the shared memory and respective vector registers of the first and second cores.

Shared scratchpad memory with parallel load-store

A system and method for producing and transmitting reduced data is disclosed. In some embodiments, the system comprises an ACF and reduction processors. The reduction processors are configured to perform a data reduction process. The ACF is configured to obtain access to input data from multiple flows, the input data identified by SGLEs included in input SGLs, and move a portion of the input data from each flow of the multiple flows to a respective reduction processor of the multiple reduction processors, such that each reduction processor receives a respective portion of the input data from each flow. The ACF is further configured to obtain access to reduced data produced from the input data using the data reduction process performed by the multiple reduction processors and move the reduced data to one or more destinations, where the reduced data identified by an output SGL.

NIC based collective acceleration

Aspects of the disclosure provide for an accelerator capable of accelerating data dependent, irregular, and/or memory-bound operations. An accelerator as described herein includes a programmable engine for efficiently executing computations on-chip that are dynamic, irregular, and/or memory-bound, in conjunction with a co-processor configured to accelerate operations that are predictable in computational load and behavior on the co-processor during design and fabrication.

Programmable accelerator for data-dependent, irregular operations

DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Direct memory access architecture with multi-level multi-striding

A system for providing memory access is disclosed. In some embodiments, the system is configured to receive at a source server fabric adapter (SFA), from a server, a memory access request comprising a virtual memory address; using associative mapping, determining whether the virtual address corresponds to a source-local memory associated with the source SFA or to a remote memory. If the virtual address corresponds to the source-local memory, the virtual memory address is translated, at the source SFA, into a physical memory address of the source-local memory. If the virtual address corresponds to the remote memory, a request message is synthesized, and the synthesized request message is transmitted to the destination SFA using a network protocol.

Transparent remote memory access over network protocol

In a system including vector registers storing right-hand side data and left-hand side data, first and second matrix staging registers, and a systolic array of processing cells for conducting matrix multiplication operations using the right-hand side data and left-hand side data, one or more processors load the right-hand side data from the vector registers to the first matrix staging register based on an instruction indicating whether to transpose the right-hand side data, load the left-hand side data from the vector registers into the second matrix staging register based on another instruction indicating whether to transpose the left-hand side data, load the right-hand side data from the first matrix staging register into the systolic array, and, in a cycle of the matrix multiplication operation, pass one or more columns of the left-hand side data from the second matrix staging register to a column of the systolic array.

Transposition of left-hand-side operand to systolic matrix multiplication

A multi-plane, multi-protocol memory switch system is disclosed. In some embodiments, a memory switch includes a plurality of switch ports, the memory switch connectable to one or more root complex (RC) devices through one or more respective switch ports of the plurality of switch ports, and the memory switch connectable to a set of endpoints through a set of other switch ports of the plurality of switch ports, wherein the set includes zero or multiple endpoints; a cacheline exchange engine configured to provide a data-exchange path between two endpoints and to map an address space of one endpoint to an address space of another endpoint; and a bulk data transfer engine configured to facilitate data-exchange between two endpoints as a source-destination data stream, one endpoint being designated a source address and another endpoint being designated a destination address.

Multi-plane, multi-protocol memory switch fabric with configurable transport

Methods, systems, and apparatus, including computer-readable media, are described for performing vector reductions using a shared scratchpad memory of a hardware circuit having processor cores that communicate with the shared memory. For each of the processor cores, a respective vector of values is generated based on computations performed at the processor core. The shared memory receives the respective vectors of values from respective resources of the processor cores using a direct memory access (DMA) data path of the shared memory. The shared memory performs an accumulation operation on the respective vectors of values using an operator unit coupled to the shared memory. The operator unit is configured to accumulate values based on arithmetic operations encoded at the operator unit. A result vector is generated based on performing the accumulation operation using the respective vectors of values.

Vector reductions using shared scratchpad memory

A computer-implemented method that includes monitoring execution of program code by first and second processor components. A computing system detects that a trigger condition is satisfied by: i) identifying an operand in a portion of the program code; or ii) determining that a current time of a clock of the computing system indicates a predefined time value. The operand and the predefined time value are used to initiate trace events. When the trigger condition is satisfied the system initiates trace events that generate trace data identifying respective hardware events occurring across the computing system. The system uses the trace data to generate a correlated set of trace data. The correlated trace data indicates a time ordered sequence of the respective hardware events. The system uses the correlated set of trace data to analyze performance of the executing program code.

Synchronous hardware event collection

Growing community of inventors

Mountain View, CA, United States of America

Thomas James Norrie