Sambanova Systems, Inc.

Greg Dykema

Kin Hing Leung

Arnav Goel

Maran Wilson

Milad Sharif

Guoyao Feng

Kuan Zhou

Taylor Lee

Tianyu Sun

Conrad Alexander Turlik

Aarti Lalwani

A coarse-grained reconfigurable processor (CGRP) system for implementing data-parallel training of a neural network is presented. The CGRP system includes a set of coarse-grained reconfigurable units (CGRUs) in a first CGRP configured to implement at least a portion of the neural network, to determine first and second gradients, respectively, of first and second model parameters based on a batch of training data, and to store the first and second gradients in a memory, a network interface including an external direct memory access (DMA) engine coupled between the memory and a network, and a work queue associated with the external DMA engine, wherein completion of determining the first gradient triggers a first work queue entry of the work queue that directs the external DMA engine to transfer the first gradient from the memory over the network to another memory coupled to a second CGRP for a gradient reduction operation.

Autonomous gradient reduction in a reconfigurable processor system

A system for a data-parallel execution of at least two implementations of an application on reconfigurable processors with different layouts is presented. The system comprises a pool of reconfigurable data flow resources with data transfer resources that interconnect first and second reconfigurable processors having first and second layouts that impose respective first and second constraints for the data-parallel execution of the application. The system further comprises an archive of configuration files and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises first and second compilers that generate for the application, based on the respective first and second constraints, first and second configuration files that are stored in the archive of configuration files and adapted to be executed data-parallel compatible on respective first and second reconfigurable processors.

System of heterogeneous reconfigurable processors for the data-parallel execution of applications

A computer-implemented method comprises a topological communications configurator (TCC) of a computing system determining a connections-optimized configuration of processors among compute nodes of the system. Processors included in the compute nodes can execute compute workers of an application of the system and can form intranodal segments of an internodal interconnection topology communicatively coupling the intranodal segments. The intranodal segments can be interconnected via an internodal interconnections fabric. The TCC can determine the connections-optimized configuration based on internodal communications costs corresponding to communications routes among the internodal segments via the internodal interconnection fabric. The computing system can comprise the TCC and can comprise a data-parallel computing system.

Determining internodal processor interconnections in a data-parallel computing system

A system for executing an application on a pool of reconfigurable processors with first and second reconfigurable processors having first and second architectures that are different from each other is presented. The system comprises an archive of configuration files with first and second configuration files for executing the application on the first and second reconfigurable processors, respectively, and a host system that is operatively coupled to the first and second reconfigurable processors. The host system comprises a runtime processor that allocates reconfigurable processors for executing the application and an auto-discovery module that is configured to perform discovery of whether the reconfigurable processors include at least one of the first reconfigurable processors and whether the reconfigurable processors include at least one of the second reconfigurable processors. The runtime processor starts execution of the first and second configuration files in dependence upon the discovery of the auto-discovery module.

System for executing an application on heterogeneous reconfigurable processors

Growing community of inventors

Palo Alto, CA, United States of America

Greg Dykema