Sambanova Systems, Inc.

Oracle America, Inc.

Ravinder Kumar

Arnav Goel

Qi Zheng

Gregory Frederick Grohoski

Sumti Jairath

Kin Hing Leung

Martin Russell Raumann

Bandish B Shah

Manish K Shah

Raghu Prabhakar

Ram Sivaramakrishnan

Ranen Chatterjee

Dawei Huang

Emre Ali Burhan

Manoj Unnikrishnan

Raghunath Shenbagam

Neal Sanghvi

Jiayu Bai

Anand Misra

Conrad Alexander Turlik

Milad Sharif

Ananda Reddy Vayyala

Paul Joseph Jordan

Mark Luttrell

Michael K Wong

Yatin Gajjar

David Alan Koeplinger

Matthew Thomas Grimm

Maran Wilson

Pushkar Shridhar Nandkar

Joshua Earle Polzin

Sudhakar Dindukurti

Steven Dai

Arjun Sabnis

A data processing system including an array of reconfigurable units and a compiler configured to generate to execute a dataflow graph of a user application is disclosed. The dataflow graph includes a sequence of temporal partitions, each temporal partition including a sequence of graph control operations. Also disclosed is an intelligent graph orchestration and execution engine (IGOEE) configured to receive an optimization objective from the complier. The optimization objective can be for minimizing execution time of the reconfigurable processor or maximizing computing resource utilization of the reconfigurable processor. The IGOEE can reorganize the sequence of temporal partitions and the sequence of graph control operations within each temporal partition to satisfy the optimization objective; and execute the reorganized dataflow graph on the reconfigurable processor. A corresponding method is also disclosed herein.

Intelligent graph execution and orchestration engine for a reconfigurable data processor

The disclosed technology relates to link-based resource allocation for a pool of reconfigurable processors. Resource allocation is provided for reconfigurable processors based on link bandwidths and link latencies. Runtime logic receives target link bandwidth and target link latency and rated link bandwidth and rated link latency. In response, the runtime logic allocates configuration files for an application, reconfigurable processors, and links between the processors. The runtime logic executes the allocated configuration files using the allocated processors and the allocated links. In another embodiment, the pool of reconfigurable processors comprise a cluster of processing nodes connected through a network.

Data processing system with link-based resource allocation for reconfigurable processors

A heterogeneous processing system and method including a host processor with a host memory having allocated buffer space, first and second processors each with memory, multiple data transfer resources, and switch and bus circuitry that communicatively couples the processors and the transfer resources. The first processor executes a first part of an application generating first data stored into the first memory. A data transfer resource is programed to transfer the first data to the buffer space and to transfer the first data from the buffer space into the second memory. The second processor executes a second part of the application generating second data stored into the second memory. The data transfer resources may include a DMA engine in which the buffer space is DMA addressable. One of the first and second processors may be a reconfigurable processor, a compute engine, or a reconfigurable dataflow unit.

Method and apparatus for data transfer between accessible memories of multiple processors in a heterogeneous processing system using two memory to memory transfer operations

A data processing system comprises a pool of reconfigurable data flow resources and a runtime processor. The pool of reconfigurable data flow resources includes arrays of physical configurable units and memory. The runtime processor includes logic to receive a plurality of configuration files for user applications. The configuration files include configurations of virtual data flow resources required to execute the user applications. The runtime processor also includes logic to allocate physical configurable units and memory in the pool of reconfigurable data flow resources to the virtual data flow resources and load the configuration files to the allocated physical configurable units. The runtime processor further includes logic to execute the user applications using the allocated physical configurable units and memory.

Runtime virtualization of reconfigurable data flow resources

A system is presented that includes a communication link, a runtime processor coupled to the communication link, and a reconfigurable processor. The reconfigurable processor is adapted for generating an interrupt to the runtime processor in response to a predetermined event and includes multiple arrays of coarse-grained reconfigurable (CGR) units and an interface to the communication link that couples the reconfigurable processor to the runtime processor via the communication link. The runtime processor is adapted for configuring the interface to the communication link to provide access to the multiple arrays of coarse-grained reconfigurable units from a physical function driver and from at least one virtual function driver, and the reconfigurable processor is adapted for sending the interrupt to the physical function driver and to a virtual function driver of the at least one virtual function driver within the runtime processor.

Handling interrupts from a virtual function in a system with a reconfigurable processor

A data processing system is presented in a client-server configuration for executing first and second applications that a client in the client-server configuration can offload for execution onto the data processing system. The data processing system includes a server and a pool of reconfigurable data flow resources that is configured to execute the first application in a first runtime context and the second application in a second runtime context. The server is configured to establish a session with the client, receive first and second execution requests for executing the first application and the second application from the client, start respective first and second execution of the first and second applications in the respective first and second runtime contexts in response to receiving the first and second execution requests, and balance a first load from the first execution with a second load from the second execution.

Load balancing system for the execution of applications on reconfigurable processors

A system is presented that includes two data processing systems that are coupled via a network, each data processing system including a reconfigurable processor with a reconfigurable processor memory, a host that is coupled to the reconfigurable processor and that includes a host processor and a host memory that is coupled to the host processor, and a network interface controller (NIC) that is operatively coupled to the reconfigurable processor and to the host processor. The reconfigurable processor of one of the data processing systems is configured to implement a virtual function that uses a virtual address for a memory access operation. An application programming interface (API) in the host processor translates the virtual address into a physical address, and the NIC uses the physical address to initiate a direct memory access operation at the reconfigurable processor memory or the host memory of the other data processing system.

Direct access to reconfigurable processor memory

A heterogeneous processing system including a host processor coupled to a host memory, a first processor coupled to a first memory, a second processor coupled to a second memory, multiple data transfer resources, and switch and bus circuitry that communicatively couples the host processor, the first processor, the second processor, and the data transfer resources. The host processor is configured to detect an application for execution by both the first processor and the second processor, to select one of multiple data transfer methods for transferring data between the first and second processors, and to configure the heterogeneous processing system based on the selected data transfer method. The data transfer methods include memory extension operation, one memory to memory transfer operation, and two memory to memory transfer operation using at least one intermediate host buffer.

Method and apparatus for selecting data access method in a heterogeneous processing system with multiple processors

A heterogeneous processing system including a host processor, a first processor with a first memory and a first data transfer resource, a second processor with a second memory, and switch and bus circuitry that communicatively couples the processors and the data transfer resource. The host processor is programmed to map virtual addresses of the second memory to physical addresses of the switch and bus circuitry and to configure the first processor to perform one memory to memory transfer operation between the first and second memories using the data transfer resource. The first processor may be configured to program the first data transfer resource. A method including mapping virtual addresses of the second memory to physical addresses of the switch and bus circuitry, and configuring the first processor to perform one memory to memory transfer operation between the first and second memories using the first data transfer resource.

Data transfer between accessible memories of multiple processors incorporated in coarse-grained reconfigurable (CGR) architecture within heterogeneous processing system using one memory to memory transfer operation

A heterogeneous processing system and method including a host processor, a first processor coupled to a first memory, a second processor coupled to a second memory, and switch and bus circuitry that communicatively couples the host processor, the first processor, and the second processor. The host processor is programmed to map virtual addresses of the second memory to physical addresses of the switch and bus circuitry and to configure the first processor to directly access the second memory using the mapped physical addresses according to memory extension operation. The first processor may be a reconfigurable processor, a reconfigurable dataflow unit, or a compute engine. The first processor may directly read data from or directly write data to the second memory while executing an application. The method may include configuring the first processor to directly access the second memory while executing an application for reading or writing data.

Method and apparatus for data access in a heterogeneous processing system with multiple processors using memory extension operation

The technology disclosed relates to interconnect-based resource allocation for reconfigurable processors. In particular, the technology disclosed relates to a runtime logic that is configured to receive target interconnect bandwidth and target interconnect latency, and rated interconnect bandwidth and rated interconnect latency. The runtime logic is further configured to respond by allocating, to configuration files defining an application graph, processing elements in a plurality of processing elements, and interconnects between the processing elements, and executing the configuration files using the allocated processing elements and the allocated interconnects.

Interconnect-based resource allocation for reconfigurable processors

A system for training parameters of a neural network includes a processing node with a processor reconfigurable at a first level of configuration granularity and a controller reconfigurable at a finer level of configuration granularity. The processor is configured to execute a first dataflow segment of the neural network with training data to generate a predicted output value using a set of neural network parameters, calculate a first intermediate result for a parameter based on the predicted output value, a target output value, and a parameter gradient, and provide the first intermediate result to the controller. The controller is configured to receive a second intermediate result over a network, and execute a second dataflow segment, dependent upon the first intermediate result and the second intermediate result, to generate a third intermediate result indicative of an update of the parameter.

Training a neural network using a non-homogenous set of reconfigurable processors

The technology disclosed relates to inter-node execution of configuration files on reconfigurable processors using network interface controller (NIC) buffers. In particular, the technology disclosed relates to a runtime logic that is configured to execute configuration files that define applications and application data for applications using a first reconfigurable processor connected to a first host, and a second reconfigurable processor connected to a second host. The first reconfigurable processor is configured to push input data for the applications in a first plurality of buffers. The first host is configured to cause a first network interface controller (NIC) to stream the input data to a second plurality of buffers from the first plurality of buffers. The second host is configured to cause a second NIC to stream the input data to the second reconfigurable processor from the second plurality of buffers.

Inter-node execution of configuration files on reconfigurable processors using network interface controller (NIC) buffers

The technology disclosed relates to runtime execution of functions across reconfigurable processor. In particular, the technology disclosed relates to a runtime logic that is configured to execute a first set of functions in a plurality of functions and/or data therefor on a first reconfigurable processor, and a second set of functions in the plurality of functions and/or data therefor on additional reconfigurable processors. Functions in the second set of functions and/or the data therefor are transmitted to the additional reconfigurable processors using one or more of a first reconfigurable processor-to-additional reconfigurable processors buffers, and results of executing the functions and/or the data therefor on the additional reconfigurable processors are transmitted to the first reconfigurable processor using one or more of additional reconfigurable processors-to-first reconfigurable processor buffers.

Runtime execution of functions across reconfigurable processor

A system for executing a graph partitioned across a plurality of reconfigurable computing units includes a processing node that has a first computing unit reconfigurable at a first level of configuration granularity and a second computing unit reconfigurable at a second, finer, level of configuration granularity. The first computing unit is configured by a host system to execute a first dataflow segment of the graph using one or more dataflow pipelines to generate a first intermediate result and to provide the first intermediate result to the second computing unit without passing through the host system. The second computing unit is configured by the host system to execute a second dataflow segment of the graph, dependent upon the first intermediate result, to generate a second intermediate result and to send the second intermediate result to a third computing unit, without passing through the host system, to continue execution of the graph.

Growing community of inventors

Fremont, CA, United States of America

Ravinder Kumar