Sambanova Systems, Inc.

Ram Sivaramakrishnan

Raghu Prabhakar

Sumti Jairath

Manish K Shah

Gregory Frederick Grohoski

Matheen Musaddiq

Kaizhao Liang

Arvind Krishna Sujeeth

Junjue Wang

Tejas Nagendra Babu Nama

Adi Fuchs

David Brian Jackson

Mark Luttrell

Kin Hing Leung

Ruddhi Arun Chaphekar

Pramod Nataraja

Ravinder Kumar

Ranen Chatterjee

Dawei Huang

Arnav Goel

Martin Russell Raumann

Bandish B Shah

Emre Ali Burhan

Manoj Unnikrishnan

Sathish Terakanambi Sheshadri

David Alan Koeplinger

Ruddhi Chaphekar

Nathan Francis Sheeley

Matthew Thomas Grimm

Weihang Fan

Disclosed is a method that includes sectioning a graph into a sequence of sections, the sequence of sections including at least a first section followed by a second section. The first section is configured to generate a first output in a first target tiling configuration in response to processing a first input in a first input tiling configuration. The graph is configured to reconfigure the first output in the first target tiling configuration to a second input in a second input tiling configuration. The second section is configured to generate a second output in a second target tiling configuration in response to processing the second input in the second input tiling configuration.

Lossless tiling in convolution networks—tiling configuration between two sections

A system includes a first coarse-grained reconfigurable (CGR) processor including first and second CGR arrays. A second CGR processor includes third and fourth CGR arrays. Each CGR array includes an address generator and coalescing unit (AGCU) designated as an array master AGCU (AMAGCU). The first CGR array AGCU is also designated as a processor master AGCU (PMAGCU) and as a system master (SMAGCU). The third CGR array AGCU is also designated as a PMAGCU. Each AMAGCU performs first execution synchronization actions, each PMAGCU also performs second execution synchronization actions, and the SMAGCU also performs third execution synchronization actions. The execution synchronization actions include a token exchange protocol for tokens such as 'array ready', “processor ready”, “system master ready”, “program start”, “array done, “processor done”, and “program terminate”. The tokens are communicated over a token bus via token interfaces that include a bus interface and a token receive FIFO.

Execution state management

Disclosed is a method that includes generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor. The plurality of output tiles of the output tensor is written in a memory, where the writing includes zero-padding the plurality of output tiles of the output tensor in the memory. The zero-padded plurality of output tiles of the output tensor are tiled, to generate a plurality of input tiles of an input tensor. The plurality of input tiles of the input tensor is processed in a second section of the processing graph.

Lossless tiling in convolution networks—padding and re-tilling at section boundaries

A system includes a coarse-grained reconfigurable (CGR) processor and a compiler configured to generate one or more configuration files for an application for execution on the CGR processor including an array of pattern compute units (PCUs) and pattern memory units (PMUs). A PCU is configured to perform an operation. A PMU comprises a plurality of data structures including a plurality of portions of operation-specific data related to the operation. The PMU is coupled to the PCU via a multi-segment datapath pipeline. The CGR processor is coupled to configure a segment of the datapath pipeline using a set of configurations bits corresponding to a portion of the operation-specific data related to the operation to activate to the segment, to further communicate the operation-specific data to the PCU via the activated segment. The CGR processor is coupled to switch among multiple PMU contexts in various segments sequentially to concurrently.

Multiple contexts for a memory unit in a reconfigurable data processor

A data processing system includes memory and reconfigurable processors, operatively coupled to the memory, configured to execute a sequence of subgraphs of a graph. The sequence of subgraphs includes a preceding subgraph and a succeeding subgraph. The data processing system also includes data flow logic, operatively coupled to the reconfigurable processors and the memory, configured to store a tiled output of the preceding subgraph as a composed input in the memory and make available parts of the composed input for processing by the succeeding subgraph.

Lossless tiling in convolution networks—data flow logic

A data processing system includes a coarse-grained reconfigurable (CGR) processor and a compiler configured to generate one or more configuration files for an application for execution on the CGR processor. The CGR processor includes an array of pattern compute units (PCUs) and pattern memory units (PMUs). A PCU comprises a plurality of single-instruction multiple data (SIMD) units configurable to form a datapath. The CGR processor is coupled to configure a datapath including a SIMD, using a set of configurations bits corresponding to an operation related to the task. The CGR processor is coupled to switch among the plurality of tasks and their corresponding PCU contexts during execution of the dataflow graph. The CGR processor is coupled to switch among tasks via static switching or dynamic switching, in response to the triggering of a task complete event generated by a preset counter, indicating completion of a current task.

Multiple contexts for a compute unit in a reconfigurable data processor

A reconfigurable processing unit is disclosed, comprising a first internal network and a second internal network with different protocols, an interface to an external network with a different protocol, a first configurable unit connected to the first internal network, a second configurable unit connected to both the first internal network and the second internal network, and a third configurable unit connected to both the second internal network and the interface to the external network. The third configurable unit is configured to receive a payload from the external network and send the transaction type identifier and the source application ID to the second configurable unit over the second internal network. The second configurable unit sends information to the first configurable unit based on the transaction type identifier and the source application ID matching the local application ID retrieved from the register.

Reconfigurable dataflow unit with streaming write functionality

A reconfigurable data processor comprises an array of configurable units and a bus system. The bus system is connected to the array of configurable units. The bus system includes a top level network and an array level network. The top level network is connected to an external data interface for communication with memory outside of the array of configurable units. The array level network is connected to configurable units in the array of configurable units.

Top level network and array level network for reconfigurable data processors

A tile of an embodiment of a coarse-grain reconfigurable architecture (CGRA) is based on an array of fused compute-memory units (FCMUs), pattern memory units (PMUs), and/or pattern compute units (PCUs) arranged in two dimensions, M×N. Unless clearly noted from context, any reference to a FCMU, PCU, or PMU may refer to one or more of the other units. The communication between a set of FCMUs is performed over a (M+1)×(N+1) switch fabric called the array-level network (ALN) where each switch has connections to its neighboring FCMUs and to neighboring switches in each of the four directions.

Dynamic destination id in an array level network of a reconfigurable dataflow processor

A data processing system receives a graph that includes a sequence of layers and executes graph cuts between a preceding layer in the graph and a succeeding layer in the graph that succeeds the preceding layer. The preceding layer generates a set of tiles on a tile-by-tile basis and the succeeding layer processes a tensor that includes multiple tiles in the set of tiles. Thus the graph is partitioned into a sequence of subgraphs, and a subgraph in the sequence of subgraphs including a sub-sequence of layers in the sequence of layers. One or more configuration files is generated to configure runtime logic to execute the sequence of subgraphs and the one or more configuration files are stored on a computer-readable media.

Lossless tiling in convolution networks—section cuts

A reconfigurable processing unit is disclosed, comprising a first internal network and a second internal network with different protocols, an interface to an external network with a different protocol, a first configurable unit sending a request to access an external memory over the first internal network, a second configurable unit receiving the request on the first internal network, obtaining a memory address, determining an identifier for the target reconfigurable processing unit, and sending the request, identifier, and memory address over the second internal network, and a third configurable unit receiving the request, identifier, and memory address on the second internal network, determining a routable address on the external network based on the identifier, synthesizing a payload with the request, address, and identifier, and sending the payload to the routable address on the external network.

Reconfigurable dataflow unit with remote read/write functionality

A computing system is disclosed, comprising a plurality of interconnected reconfigurable dataflow units (RDUs). Each RDU includes configurable units, internal networks, and external interfaces. The first configurable unit of the first RDU sends a request to access an external memory attached to the second RDU over its first internal network. The second configurable unit of the first RDU obtains a memory address for the request, determines an identifier for the second RDU, and sends the request, identifier, and memory address to the third configurable unit of the first RDU over its second internal network. The third configurable unit of the first RDU generates a routable address on the external network, synthesizes a payload, and sends it through an external network interface. The third configurable unit of the second RDU receives the payload, and the fourth configurable unit of the second RDU uses the address to access the external memory.

Peer-to-peer communication between reconfigurable dataflow units

A data processing system includes compile time logic to section a graph into a sequence of sections, including a first section followed by a second section. The compile time logic configured the first section to generate a first output in a first non-overlapping target configuration in response to processing an input in a first overlapping input configuration, and configures the second section to generate a second output in a second non-overlapping target configuration in response to processing the first output in a second overlapping input configuration. The compile time logic also creates a set of computer instructions to execute the first section and the second section on a target processing system.

Lossless tiling in convolution networks—resetting overlap factor to zero at section boundaries

Disclosed is a data processing system that includes a plurality of reconfigurable processors and processor memory. Runtime logic, operatively coupled to the plurality of reconfigurable processors and the processor memory, is configured to configure at least one reconfigurable processor in the plurality of reconfigurable processors with a first subgraph in a sequence of subgraphs of a graph; load an input onto the processor memory; on a tile-by-tile basis, process a first set of input tiles from the input through the first subgraph and generate a first set of intermediate tiles, load the first set of intermediate tiles onto the processor memory, and process the first set of intermediate tiles through the first subgraph and generate a first set of output tiles; and compose output tiles in the first set of output tiles into a first composed input, and load the first composed input onto the processor memory.

Lossless tiling in convolution networks—materialization of tensors

A compiler transforms a high-level program into configuration data for a coarse-grained reconfigurable (CGR) data processor with an array of CGR units. The compiler includes a method that identifies a skip buffer in a dataflow graph, determines limitations associated with the array, and searches for a lowest cost implementation topology and stage depth. At least three topologies are considered, including a cascaded buffer topology, a hybrid buffer topology, and a striped buffer topology. The lowest cost implementation topology and stage depth are based on the size of the buffered data (usually, the size of a tensor), the depth of the skip buffer, and the array's limitations. The hybrid buffer topology includes multiple sections of parallel memory units. The data travels between memory units in one section to adjacent memory units in a next section without intervening reorder buffers.

Growing community of inventors

San Jose, CA, United States of America

Ram Sivaramakrishnan