Tenstorrent Inc.

Altera Corporation

Tenstorrent Ai Ulc

Intel Corporation

Davor Capalija

Ljubisa Bajic

Jasmina Vasiljevic

Ivan Matosevic

Alex Cejkov

Utku Aydonat

S Alexander Chin

Andrew Ling

Tomasz Czajkowski

Andrei Mihai Hagiescu Miriste

Hassan Farooq

Andrew Grebenisan

Yu Ting Chen

Stephen Chin

Vladimir Blagojevic

Almeet Bhullar

Akhmed Rakhmati

Djordje Maksimovic

Andrew Lewycky

Eriko Nurvitadhi

Deborah T Marr

John Stuart Freeman

Milos Trajkovic

Stanislav Sokorac

Alan Baker

Meghan Lele

Zahi Moudallal

Jack Z Yinger

Joseph Chu

Andrew Chaang Ling

Methods and systems for the accelerated execution of a directed graph are disclosed. The execution can involve the generation of an inference from a set of inputs provided to an artificial neural network. In a specific example, a method for executing a directed graph includes receiving at least two batches of indices. The batches of indices, when used to access a set of embeddings, provide at least two batches of embedding outputs and execute a layer of the directed graph. The method further includes accessing the set of embeddings using the at least two batches of indices. The method further includes rearranging, based on a set of latencies for the accessing step, the at least two batches of embedding outputs into at least two batches of rearranged embeddings. The method further includes providing the at least two batches of rearranged embeddings to a subsequent layer of the directed graph.

Graph execution using access request response dynamic batch assembly

Methods and systems relating to the field of parallel computing are disclosed herein. The methods and systems disclosed include approaches for sparsity uniformity enforcement for a set of computational nodes which are used to execute a complex computation. A disclosed method includes determining a sparsity distribution in a set of operand data, and generating, using a compiler, a set of instructions for executing, using the set of operand data and a set of processing cores, a complex computation. Alternatively, the method includes altering the operand data. The method also includes distributing the set of operand data to the set of processing cores for use in executing the complex computation in accordance with the set of instructions. Either the altering is conducted to, or the compiler is programmed to, balance the sparsity distribution among the set of processing cores.

Sparsity uniformity enforcement for multicore processor

Methods and systems related to the efficient execution of complex computations by a multicore processor and the movement of data among the various processing cores in the multicore processor are disclosed. A multicore processor includes a set of processing cores and associated sets of processing pipelines, core controllers, routers, and network interface units. The multicore processor also includes a computation layer, for conducting computations using the set of processing cores, with executable instructions for the set of processing pipelines which are executed by the set of core controllers. The multicore processor also includes a network-on-chip layer, for connecting the set of processing cores in the multicore processor, with executable instructions for the set of routers and the set of network interface units. The multicore processor also includes a set of programmable controllers, with executable instructions for reformatting computational data from the computation layer for transmission through the network-on-chip layer.

Overlay layer for network of processor cores

Methods and systems for executing an application data flow graph on a set of computational nodes are disclosed. The computational nodes can each include a programmable controller from a set of programmable controllers, a memory from a set of memories, a network interface unit from a set of network interface units, and an endpoint from a set of endpoints. A disclosed method comprises configuring the programmable controllers with instructions. The method also comprises independently and asynchronously executing the instructions using the set of programmable controllers in response to a set of events exchanged between the programmable controllers themselves, between the programmable controllers and the network interface units, and between the programmable controllers and the set of endpoints. The method also comprises transitioning data in the set of memories on the computational nodes in accordance with the application data flow graph and in response to the execution of the instructions.

Overlay layer hardware unit for network of processor cores

Methods and systems relating to computational circuitry are disclosed herein. A disclosed computational circuit includes a math circuit, a first accumulator, and a second accumulator. The first accumulator has a first memory. The second accumulator has a second memory. The first accumulator is communicatively connected to the math circuit and accumulates values from the math circuit in the first memory. The second accumulator is communicatively connected to the first memory and accumulates values from the first memory in the second memory. The first memory is faster and smaller than the second memory.

Computational circuit with hierarchical accumulator

Methods and systems associated with caches are disclosed. One disclosed system includes at least one memory storing at least two data structures. The at least two data structures include a first data structure and a second data structure. The system also includes at least two caches with a first cache which caches the first data structure and a second cache which caches the second data structure. The system also includes a controller communicatively coupled to the at least two caches. The controller separately configures the first cache based on the first data structure and the second cache based on the second data structure. The system also comprises at least one processor communicatively coupled to the at least two caches. The processor accesses each of the at least two data structures using the at least two caches and during the execution of a complex computation.

Data structure optimized dedicated memory caches

Methods and systems related to parallel computing using heterogeneous networks of computational nodes are disclosed herein. A method for executing a complex computation on a heterogeneous set of computational nodes linked together by a set of links in a network is disclosed. The method includes compiling, using a table of bandwidth values for the set of links in the network, a set of instructions for routing data for the execution of the complex computation. The method also includes configuring a set of programmable controllers on the heterogeneous set of computational nodes with the set of instructions. The method also includes executing the set of instructions using the set of programmable controllers. The method also includes routing data through the network to facilitate the execution of the complex computation by the heterogeneous set of computational nodes and in response to the execution of the instructions.

Seamless place and route for heterogenous network of processor cores

Methods and systems for executing an application data flow graph using a network of computational nodes are disclosed. In specific examples, the network of computational nodes can be a network-on-chip for a multicore processor. One method includes transitioning first application data from a first source computational node to an intermediary computational node. The method can also include providing second application data, from a computation layer of the network of computational nodes, on the intermediary computational node. The method can also include multicasting the first application data in combination with the second application data from the intermediary computational node to at least two destination computational nodes. The first source computational node, the intermediary computational node, and the at least two destination computational nodes are all in the network of computational nodes.

Application data flow graph execution using network-on-chip overlay

Processor cores using packet identifiers for routing and computation are disclosed. One method includes executing a complex computation using a set of processing cores. The method includes routing a set of packets using a set of packet identifiers and executing a set of instructions. The set of instructions are defined using a set of operand identifiers. The operand identifiers represent packet identifiers in the set of packet identifiers. In specific implementations the set of the operand identifiers represent packet identifiers in the set of packet identifiers in that a set of memories on the set of processing cores stores data values in common association with both the set of packets, and a set of operands identified by the set of operand identifiers. In specific implementations the set of operand identifiers and packet identifiers are unambiguously mapped to an underlying set of application datums of the complex computation.

Processor cores using packet identifiers for routing and computation

Methods and systems related to the efficient execution of complex computations by a multicore processor and the movement of data among the various processing cores in the multicore processor are disclosed. A multicore processor stack for the multicore processor can include a computation layer, for conducting computations using the processing cores in the multicore processor, with executable instructions for processing pipelines in the processing cores. The multicore processor stack can also include a network-on-chip layer, for connecting the processing cores in the multicore processor, with executable instructions for routers and network interface units in the multicore processor. The computation layer and the network-on-chip layer can be logically isolated by a network-on-chip overlay layer.

Matrix multiplication systolic array feed methods and related processing element (PE) microarchitectures for efficiently implementing systolic array generic matrix multiplier (SGEMM) in integrated circuits is provided. A systolic array architecture may include a processing element array, a column feeder array, and a row feeder array. A bandwidth of external memory may be reduced by a factor of reduction based on interleaving of the matrix data via a feeding pattern of the column feeder array and the row feeder array.

Memory-size- and bandwidth-efficient method for feeding systolic array matrix multipliers

Growing community of inventors

Toronto, Canada

Davor Capalija