Sambanova Systems, Inc.

Qi Zheng

Ravinder Kumar

Arnav Goel

Neal Sanghvi

Jiayu Bai

Gregory Frederick Grohoski

Sumti Jairath

Kin Hing Leung

Martin Russell Raumann

Bandish B Shah

Raghunath Shenbagam

Anand Misra

Conrad Alexander Turlik

Ananda Reddy Vayyala

Maran Wilson

Milad Sharif

Pushkar Shridhar Nandkar

Joshua Earle Polzin

Arjun Sabnis

A data processing system including an array of reconfigurable units and a compiler configured to generate to execute a dataflow graph of a user application is disclosed. The dataflow graph includes a sequence of temporal partitions, each temporal partition including a sequence of graph control operations. Also disclosed is an intelligent graph orchestration and execution engine (IGOEE) configured to receive an optimization objective from the complier. The optimization objective can be for minimizing execution time of the reconfigurable processor or maximizing computing resource utilization of the reconfigurable processor. The IGOEE can reorganize the sequence of temporal partitions and the sequence of graph control operations within each temporal partition to satisfy the optimization objective; and execute the reorganized dataflow graph on the reconfigurable processor. A corresponding method is also disclosed herein.

Intelligent graph execution and orchestration engine for a reconfigurable data processor

A method of pipelining execution stages of a pipelined application can comprise a Buffer Pipeline Manager (BPM) of a Buffer Pipelined Application computing System (BPAS) allocating pipeline buffers, configuring access to the pipeline buffers by stage processors of the system, transferring buffers from one stage processor to a successor stage processor, and transferring data from a buffer in one memory to a buffer in an alternative memory. The BPM can allocate the buffers based on execution parameters associated with the pipelined application and/or stage processors. The BPM can transfer data to a buffer in an alternative memory based on performance, capacity, and/or topological attributes of the memories and/or processors utilizing the memories. The BPM can perform operations of the method responsive to interfaces of a Pipeline Programming Interface (PPI). A BPAS can comprise hardware processors, physical memories, stage processors, an application execution program, the PPI, and the BPM.

System and method for user interactive pipelining of a computing application

A heterogeneous processing system and method including a host processor with a host memory having allocated buffer space, first and second processors each with memory, multiple data transfer resources, and switch and bus circuitry that communicatively couples the processors and the transfer resources. The first processor executes a first part of an application generating first data stored into the first memory. A data transfer resource is programed to transfer the first data to the buffer space and to transfer the first data from the buffer space into the second memory. The second processor executes a second part of the application generating second data stored into the second memory. The data transfer resources may include a DMA engine in which the buffer space is DMA addressable. One of the first and second processors may be a reconfigurable processor, a compute engine, or a reconfigurable dataflow unit.

Method and apparatus for data transfer between accessible memories of multiple processors in a heterogeneous processing system using two memory to memory transfer operations

A data processing system comprises a pool of reconfigurable data flow resources and a runtime processor. The pool of reconfigurable data flow resources includes arrays of physical configurable units and memory. The runtime processor includes logic to receive a plurality of configuration files for user applications. The configuration files include configurations of virtual data flow resources required to execute the user applications. The runtime processor also includes logic to allocate physical configurable units and memory in the pool of reconfigurable data flow resources to the virtual data flow resources and load the configuration files to the allocated physical configurable units. The runtime processor further includes logic to execute the user applications using the allocated physical configurable units and memory.

Runtime virtualization of reconfigurable data flow resources

A data processing system is presented in a client-server configuration for executing first and second applications that a client in the client-server configuration can offload for execution onto the data processing system. The data processing system includes a server and a pool of reconfigurable data flow resources that is configured to execute the first application in a first runtime context and the second application in a second runtime context. The server is configured to establish a session with the client, receive first and second execution requests for executing the first application and the second application from the client, start respective first and second execution of the first and second applications in the respective first and second runtime contexts in response to receiving the first and second execution requests, and balance a first load from the first execution with a second load from the second execution.

Load balancing system for the execution of applications on reconfigurable processors

A heterogeneous processing system including a host processor coupled to a host memory, a first processor coupled to a first memory, a second processor coupled to a second memory, multiple data transfer resources, and switch and bus circuitry that communicatively couples the host processor, the first processor, the second processor, and the data transfer resources. The host processor is configured to detect an application for execution by both the first processor and the second processor, to select one of multiple data transfer methods for transferring data between the first and second processors, and to configure the heterogeneous processing system based on the selected data transfer method. The data transfer methods include memory extension operation, one memory to memory transfer operation, and two memory to memory transfer operation using at least one intermediate host buffer.

Method and apparatus for selecting data access method in a heterogeneous processing system with multiple processors

A heterogeneous processing system including a host processor, a first processor with a first memory and a first data transfer resource, a second processor with a second memory, and switch and bus circuitry that communicatively couples the processors and the data transfer resource. The host processor is programmed to map virtual addresses of the second memory to physical addresses of the switch and bus circuitry and to configure the first processor to perform one memory to memory transfer operation between the first and second memories using the data transfer resource. The first processor may be configured to program the first data transfer resource. A method including mapping virtual addresses of the second memory to physical addresses of the switch and bus circuitry, and configuring the first processor to perform one memory to memory transfer operation between the first and second memories using the first data transfer resource.

Data transfer between accessible memories of multiple processors incorporated in coarse-grained reconfigurable (CGR) architecture within heterogeneous processing system using one memory to memory transfer operation

A heterogeneous processing system and method including a host processor, a first processor coupled to a first memory, a second processor coupled to a second memory, and switch and bus circuitry that communicatively couples the host processor, the first processor, and the second processor. The host processor is programmed to map virtual addresses of the second memory to physical addresses of the switch and bus circuitry and to configure the first processor to directly access the second memory using the mapped physical addresses according to memory extension operation. The first processor may be a reconfigurable processor, a reconfigurable dataflow unit, or a compute engine. The first processor may directly read data from or directly write data to the second memory while executing an application. The method may include configuring the first processor to directly access the second memory while executing an application for reading or writing data.

Method and apparatus for data access in a heterogeneous processing system with multiple processors using memory extension operation

A system for training parameters of a neural network includes a processing node with a processor reconfigurable at a first level of configuration granularity and a controller reconfigurable at a finer level of configuration granularity. The processor is configured to execute a first dataflow segment of the neural network with training data to generate a predicted output value using a set of neural network parameters, calculate a first intermediate result for a parameter based on the predicted output value, a target output value, and a parameter gradient, and provide the first intermediate result to the controller. The controller is configured to receive a second intermediate result over a network, and execute a second dataflow segment, dependent upon the first intermediate result and the second intermediate result, to generate a third intermediate result indicative of an update of the parameter.

Training a neural network using a non-homogenous set of reconfigurable processors

A system for executing a graph partitioned across a plurality of reconfigurable computing units includes a processing node that has a first computing unit reconfigurable at a first level of configuration granularity and a second computing unit reconfigurable at a second, finer, level of configuration granularity. The first computing unit is configured by a host system to execute a first dataflow segment of the graph using one or more dataflow pipelines to generate a first intermediate result and to provide the first intermediate result to the second computing unit without passing through the host system. The second computing unit is configured by the host system to execute a second dataflow segment of the graph, dependent upon the first intermediate result, to generate a second intermediate result and to send the second intermediate result to a third computing unit, without passing through the host system, to continue execution of the graph.

Executing a neural network graph using a non-homogenous set of reconfigurable processors

A method for executing applications in a system comprising general hardware and reconfigurable hardware includes accessing a first execution file comprising metadata storing a first priority indicator associated with a first application, and a second execution file comprising metadata storing a second priority indicator associated with a second application. In an example, use of the reconfigurable hardware is interleaved between the first application and the second application, and the interleaving is scheduled to take into account (i) workload of the reconfigurable hardware and (ii) the first priority indicator and the second priority indicator associated with the first application and the second application, respectively. In an example, when the reconfigurable hardware is used by one of the first and second applications, the general hardware is used by another of the first and second applications.

Time-multiplexed use of reconfigurable hardware

Roughly described, the invention involves a system including a plurality of functional units that execute different segments of a dataflow, and share intermediate results via a peer-to-peer messaging protocol. The functional units are reconfigurable, with different units being reconfigurable at different levels of granularity. The peer-to-peer messaging protocol includes control tokens or other mechanisms by which the consumer of the intermediate results learns that data has been transferred, and in response thereto triggers its next dataflow segment. A host or configuration controller configures the data units with their respective dataflow segments, but once execution of the configured dataflow begins, no host need be involved in orchestrating data synchronization, the transfer of intermediate results, or the triggering of processing after the data are received. Control overhead is therefore minimized.

Dataflow function offload to reconfigurable processors

Roughly described, a system for data parallel training of a neural network on multiple reconfigurable units configured by a host with dataflow pipelines to perform different steps in the training CGRA units are configured to evaluate first and second sequential sections of neural network layers based on a respective subset of training data, and to back-propagate the error through the sections to calculate parameter gradients for the respective subset. Gradient synchronization and reduction are performed by one or more units having finer grain reconfigurability, such as an FPGA. The FPGA performs synchronization and reduction of the gradients for the second section while the CGRA units perform back-propagation through the first sequential section. Intermediate results are transmitted using a P2P message passing protocol layer. Execution of dataflow segments in the different units is triggered by receipt of data, rather than by a command from any host system.

Growing community of inventors

Fremont, CA, United States of America

Qi Zheng