Habana Labs Ltd.

Doron Singer

Ron Shalev

Amos Goldman

Michael Zuckerman

Tzachi Cohen

Shlomo Raikin

Oren Kaidar

Eyal Radiano

An apparatus includes a memory and one or more processors. The memory is configured to store tensors for Machine Learning (ML) processing. The one or more processors are configured to receive a work plan associated with a subgraph of a ML graph of a ML model, the work plan supports processing of tensors having respective shapes in a selected range of shapes. A shape of a tensor specifies respective sizes of dimensions of that tensor. The one or more processors are further configured to receive from the memory an input tensor having an actual shape, to modify the work plan based on the actual shape to produce a modified work plan for processing the input tensor in accordance with the subgraph, and to process the input tensor in accordance with the subgraph by submitting the modified work plan for execution by one or more of the processors.

Machine learning with variable-shape tensors

A method for computing includes defining a processing pipeline, including at least a first stage in which producer processors compute and output data to respective locations in a buffer and a second processing stage in which one or more consumer processors read the data from the buffer and apply a computational task to the data read from the buffer. The computational task is broken into multiple, independent work units, for application by the consumer processors to respective ranges of the data in the buffer, and respective indexes are assigned to the work units in a predefined index space. A mapping is generated between the index space and the addresses in the buffer, and execution of the work units is scheduled such that at least one of the work units can begin execution before all the producer processors have completed the first processing stage.

Fine-grained pipelining using index space mapping

A method for computing includes providing software source code defining a processing pipeline including multiple, sequential stages of parallel computations, in which a plurality of processors apply a computational task to data read from a buffer. A static code analysis is applied to the software source code so as to break the computational task into multiple, independent work units, and to define an index space in which the work units are identified by respective indexes. Based on the static code analysis, mapping parameters that define a mapping between the index space and addresses in the buffer are computed, indicating by the mapping the respective ranges of the data to which the work units are to be applied. The source code is compiled so that the processors execute the work units identified by the respective indexes while accessing the data in the buffer in accordance with the mapping.

Index space mapping using static code analysis

Growing community of inventors

Tel Aviv, Israel

Doron Singer