Tenstorrent Inc.

Tenstorrent Ai Ulc

Alex Cejkov

Ljubisa Bajic

Lejla Bajic

Davor Capalija

Ivan Matosevic

Jasmina Vasiljevic

Milos Trajkovic

Utku Aydonat

S Alexander Chin

Andrew Lewycky

Processing cores with data associative adaptive rounding and associated methods are disclosed herein. One disclosed processing core comprises an arithmetic logic unit cluster configured to generate a value for a unit of directed graph data using input directed graph data, a comparator coupled to a threshold register and a data register, a core controller configured to load a threshold value into the threshold register when the value for the unit of directed graph data is loaded into the data register, and a rounding circuit. The rounding circuit is configured to receive the value for the unit of directed graph data from the arithmetic logic unit cluster and conditionally round the value for the unit of directed graph data based on a comparator output from the comparator.

Processing core with data associative adaptive rounding

Methods and systems for the accelerated execution of a directed graph are disclosed. The execution can involve the generation of an inference from a set of inputs provided to an artificial neural network. In a specific example, a method for executing a directed graph includes receiving at least two batches of indices. The batches of indices, when used to access a set of embeddings, provide at least two batches of embedding outputs and execute a layer of the directed graph. The method further includes accessing the set of embeddings using the at least two batches of indices. The method further includes rearranging, based on a set of latencies for the accessing step, the at least two batches of embedding outputs into at least two batches of rearranged embeddings. The method further includes providing the at least two batches of rearranged embeddings to a subsequent layer of the directed graph.

Graph execution using access request response dynamic batch assembly

Methods and systems related to the efficient execution of complex computations by a multicore processor and the movement of data among the various processing cores in the multicore processor are disclosed. A multicore processor includes a set of processing cores and associated sets of processing pipelines, core controllers, routers, and network interface units. The multicore processor also includes a computation layer, for conducting computations using the set of processing cores, with executable instructions for the set of processing pipelines which are executed by the set of core controllers. The multicore processor also includes a network-on-chip layer, for connecting the set of processing cores in the multicore processor, with executable instructions for the set of routers and the set of network interface units. The multicore processor also includes a set of programmable controllers, with executable instructions for reformatting computational data from the computation layer for transmission through the network-on-chip layer.

Overlay layer for network of processor cores

Methods and systems relating to computational circuitry are disclosed herein. A disclosed computational circuit includes a math circuit, a first accumulator, and a second accumulator. The first accumulator has a first memory. The second accumulator has a second memory. The first accumulator is communicatively connected to the math circuit and accumulates values from the math circuit in the first memory. The second accumulator is communicatively connected to the first memory and accumulates values from the first memory in the second memory. The first memory is faster and smaller than the second memory.

Computational circuit with hierarchical accumulator

Methods and systems associated with caches are disclosed. One disclosed system includes at least one memory storing at least two data structures. The at least two data structures include a first data structure and a second data structure. The system also includes at least two caches with a first cache which caches the first data structure and a second cache which caches the second data structure. The system also includes a controller communicatively coupled to the at least two caches. The controller separately configures the first cache based on the first data structure and the second cache based on the second data structure. The system also comprises at least one processor communicatively coupled to the at least two caches. The processor accesses each of the at least two data structures using the at least two caches and during the execution of a complex computation.

Data structure optimized dedicated memory caches

Methods and systems regarding the rapid and efficient compression and decompression of sparse data are disclosed. One method for compressing a set of data from a sparse matrix includes, evaluating a sequence of data entries from the set of data, extracting a sequence of sparse data values from the sequence, extracting a sequence of non-sparse data value run lengths from the sequence, formulating a set of row pointers from the sequence, storing the sequence of sparse data values in a first set of memory addresses, and storing the sequence of non-sparse data value run lengths in a second set of memory addresses. The set of row pointers identify a set of rows of the sparse matrix in both the first and second sets of memory addresses. Rapid decompression can be conducted using the row pointers.

Growing community of inventors

Toronto, Canada

Alex Cejkov