Sambanova Systems, Inc.

Oracle International Corporation

Oracle America, Inc.

Sumti Jairath

Raghu Prabhakar

Ram Sivaramakrishnan

Gregory Frederick Grohoski

Kin Hing Leung

Manish K Shah

Arvind Krishna Sujeeth

Matheen Musaddiq

Junjue Wang

Ruddhi Arun Chaphekar

Tejas Nagendra Babu Nama

Kaizhao Liang

Adi Fuchs

Ravinder Kumar

Martin Russell Raumann

Bandish B Shah

Mark Luttrell

Pramod Nataraja

Ranen Chatterjee

Sitanshu Gupta

Dawei Huang

Arnav Goel

Emre Ali Burhan

Manoj Unnikrishnan

David Brian Jackson

David Alan Koeplinger

Zhuo Chen

Qi Zheng

Matthew Thomas Grimm

Ajit Punj

Hans Eberle

Nils Gura

Kathirgamar Aingaran

Wladyslaw Olesinski

Yuan Lin

Nathan Francis Sheeley

Robert Dickson

Aron J Silverton

Chen Liu

Peter J Yakutis

Luca Boasso

Bowen Yang

Garret Frederick Swart

Paul N Loewenstein

William Havinden Bridge, Jr

Ali Vahidsafa

John Johnson

Weiwei Chen

David Smentek

Andrew Deng

Scott Layson Burson

Blaine Rister

Serena Wing Yee Leung

Hong Suh

Qingjian Li

Ruddhi Chaphekar

Yue Fu

Sebastian Turullols

Thomas M Wicki

John R Feehrer

Ramaswamy Sivaramakrishnan

Sanjiv Kapil

Fei Wang

Manling Yang

Joshua Brot

Amitabh Menon

Aings Aingaran

Fred Han-Ching Tsai

Likun Hao

Venkat Krishna Srinivasan

John G Johnson

William H Bridge

Disclosed is a method that includes generating by an output processing node of a first section of a processing graph, a plurality of output tiles of an output tensor. The plurality of output tiles of the output tensor is written in a memory, where the writing includes zero-padding the plurality of output tiles of the output tensor in the memory. The zero-padded plurality of output tiles of the output tensor are tiled, to generate a plurality of input tiles of an input tensor. The plurality of input tiles of the input tensor is processed in a second section of the processing graph.

Lossless tiling in convolution networks—padding and re-tilling at section boundaries

A system with a cost estimation tool for estimating a realized bandwidth consumption of a logical edge between a logical producer unit and a logical consumer unit of an operation unit graph during placement and routing of the logical producer unit, the logical consumer unit, and the logical edge onto a reconfigurable processor is presented as well as a method of operating such a cost estimation tool and a non-transitory computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to operate such a cost estimation tool The cost estimation tool may be configured to determine the realized bandwidth consumption of the tentative assignment based on an upper bandwidth limit of the logical edge, an end-to-end bandwidth, a scaling factor of a realized bandwidth, and a congestion estimation of the physical link.

Operating a cost estimation tool for placing and routing an operation unit graph on a reconfigurable processor

In a method, in response to an interface a computer-implemented analysis assistant initiates a presentation of inefficiency results, determined an efficiency analyzer based on a mapping of a dataflow program to execute on hardware of a computing system. The assistant receives an inefficiency included among the inefficiency results and composes formatted inefficiency results comprising a presentation format of the inefficiency to assist a developer of the dataflow program to interpret the inefficiency. The analysis assistant outputs the formatted inefficiency results to an interface, which can comprise an interface to output the formatted inefficiency results for use by the developer to improve the dataflow program in association with the inefficiency. In implementations the presentation can comprise an interactive presentation with a developer of the dataflow program. A computer program product and a computing system can implement the method. The computing system can execute the assistant, and can include the interfaces.

Analysis assistant for determining execution inefficiencies in dataflow programs

In a method a computer-implemented efficiency analyzer selects operators from an intermediate representation of a dataflow program. The operators are included in a mapping of the operators to hardware of a computing system to execute the dataflow program. Based on the mapping and a description of the hardware, the efficiency analyzer computes an execution metric associated with executing the operators on the hardware. Based on the execution metric and hardware description, the efficiency analyzer determines an inefficiency metric, and based on the inefficiency metric, the efficiency analyzer determines an inefficiency associated with the dataflow program. The computing system to execute the dataflow program can comprise a coarse grain computing system and the hardware can include a reconfigurable processor of the computing system. A computer program product and a computing system to a the dataflow program can implement the method.

Method and system to determine execution inefficiencies in dataflow programs

According to a computing method a compiler determines a recompute node included in a dataflow application and a checkpoint tensor produced by the recompute node. The compiler determines a recompute cost to recompute the checkpoint tensor, and a memory cost to checkpoint the checkpoint tensor in a memory. Based on the recompute cost and/or the memory cost, the compiler determines a solution cost and compares the solution cost to a solution threshold. Based on comparing the solution cost to the solution threshold, the compiler determines a checkpoint solution to execute the dataflow application. The checkpoint solution can comprise recomputing or checkpointing the checkpoint tensor. In some implementations, the compiler can determine a recompute ratio of the recompute cost to the memory cost and can compare the recompute ratio to the solution threshold. A computer program product and a computing system can implement aspects of the method.

Tensor checkpoint optimization in dataflow computing applications

A cost estimation tool in a system for implementing an operation unit graph on a reconfigurable processor is presented as well as a method of operating a cost estimation tool for determining scaled logical edge bandwidths in an operation unit graph in preparation of placing and routing the operation unit graph onto a reconfigurable processor. The cost estimation tool may be configured to receive the operation unit graph, divide the operation unit graph in first and second subgraphs, determine maximum latencies of the first and second subgraphs, and determine a scaled logical edge bandwidth of a logical edge that couples a first logical unit of M logical units in the first subgraph with a second logical unit of N logical units in the first subgraph based on M, N, and scaled bandwidth limits of the M and N logical units.

Estimating a scaled cost of implementing an operation unit graph on a reconfigurable processor

A sorting tool for determining an ordered sequence of nodes in an operation unit graph for placing and routing the operation unit graph onto a reconfigurable processor is presented as well as a method of operating a sorting tool for determining an ordered sequence of nodes in an operation unit graph for placing and routing the operation unit graph onto a reconfigurable processor. The sorting tool is configured to receive the operation unit graph including a set of unsorted nodes and edges that interconnect nodes in the set of unsorted nodes, determine an ordered sequence of the nodes in the operation unit graph, and provide the ordered sequence of nodes for the placing and routing of the operation unit graph onto the reconfigurable processor.

Sorting the nodes of an operation unit graph for implementation in a reconfigurable processor

A data processing system includes memory and reconfigurable processors, operatively coupled to the memory, configured to execute a sequence of subgraphs of a graph. The sequence of subgraphs includes a preceding subgraph and a succeeding subgraph. The data processing system also includes data flow logic, operatively coupled to the reconfigurable processors and the memory, configured to store a tiled output of the preceding subgraph as a composed input in the memory and make available parts of the composed input for processing by the succeeding subgraph.

Lossless tiling in convolution networks—data flow logic

A reconfigurable data processor comprises an array of configurable units and a bus system. The bus system is connected to the array of configurable units. The bus system includes a top level network and an array level network. The top level network is connected to an external data interface for communication with memory outside of the array of configurable units. The array level network is connected to configurable units in the array of configurable units.

Top level network and array level network for reconfigurable data processors

A placer and router for an iterative placement and routing of a sorted operation unit graph on a reconfigurable processor is presented as well as a method of operating a placer and router for an iterative placement and routing of a sorted operation unit graph on a reconfigurable processor. The placer and router is configured to receive an architectural specification of the reconfigurable processor and the sorted operation unit graph having an ordered sequence of nodes and edges that interconnect nodes in the ordered sequence of nodes. The placer and router is further configured to iteratively assign nodes of the sorted operation unit graph to locations on the reconfigurable processor followed by an assignment of edges that connect nodes that were assigned in the current iteration and nodes that were assigned in previous iterations to interconnection resources of the reconfigurable processor.

Routing method based on a sorted operation unit graph for an iterative placement and routing on a reconfigurable processor

A tile of an embodiment of a coarse-grain reconfigurable architecture (CGRA) is based on an array of fused compute-memory units (FCMUs), pattern memory units (PMUs), and/or pattern compute units (PCUs) arranged in two dimensions, M×N. Unless clearly noted from context, any reference to a FCMU, PCU, or PMU may refer to one or more of the other units. The communication between a set of FCMUs is performed over a (M+1)×(N+1) switch fabric called the array-level network (ALN) where each switch has connections to its neighboring FCMUs and to neighboring switches in each of the four directions.

Dynamic destination id in an array level network of a reconfigurable dataflow processor

The technology disclosed relates to storing a dataflow graph with a plurality of compute nodes that transmit data along data connections, and controlling data transmission between compute nodes in the plurality of compute nodes along the data connections by using control connections to control writing of data.

Flow control for reconfigurable processors

A data processing system receives a graph that includes a sequence of layers and executes graph cuts between a preceding layer in the graph and a succeeding layer in the graph that succeeds the preceding layer. The preceding layer generates a set of tiles on a tile-by-tile basis and the succeeding layer processes a tensor that includes multiple tiles in the set of tiles. Thus the graph is partitioned into a sequence of subgraphs, and a subgraph in the sequence of subgraphs including a sub-sequence of layers in the sequence of layers. One or more configuration files is generated to configure runtime logic to execute the sequence of subgraphs and the one or more configuration files are stored on a computer-readable media.

Lossless tiling in convolution networks—section cuts

A data processing system includes compile time logic to section a graph into a sequence of sections, including a first section followed by a second section. The compile time logic configured the first section to generate a first output in a first non-overlapping target configuration in response to processing an input in a first overlapping input configuration, and configures the second section to generate a second output in a second non-overlapping target configuration in response to processing the first output in a second overlapping input configuration. The compile time logic also creates a set of computer instructions to execute the first section and the second section on a target processing system.

Lossless tiling in convolution networks—resetting overlap factor to zero at section boundaries

Disclosed is a data processing system that includes a plurality of reconfigurable processors and processor memory. Runtime logic, operatively coupled to the plurality of reconfigurable processors and the processor memory, is configured to configure at least one reconfigurable processor in the plurality of reconfigurable processors with a first subgraph in a sequence of subgraphs of a graph; load an input onto the processor memory; on a tile-by-tile basis, process a first set of input tiles from the input through the first subgraph and generate a first set of intermediate tiles, load the first set of intermediate tiles onto the processor memory, and process the first set of intermediate tiles through the first subgraph and generate a first set of output tiles; and compose output tiles in the first set of output tiles into a first composed input, and load the first composed input onto the processor memory.

Lossless tiling in convolution networks—materialization of tensors

Growing community of inventors

Santa Clara, CA, United States of America

Sumti Jairath