Microsoft Technology Licensing, LLC

Intel Corporation

Gray Research LLC

Jan Stephen Gray

Ali-Reza Adl-Tabatabai

Gad Shlomo Sheaffer

David S Callahan

Martin Taillefer

Bratin Saha

Aaron L Smith

Vadim Bassin

Burton Jordan Smith

Douglas Christopher Burger

Yosseff Levanoni

Vinod Grover

James R Larus

Arun Upadhyaya Kishan

Koichi Yamada

Timothy Lawrence Harris

Landy Wang

Rodney Thomas Limprecht

John Joseph Duffy

Michael M Magruder

Christopher Wellington Brumme

David L Detlefs

Eric S Chung

Yang Ni

Dave Detlefs

Mohsen M Al-Ghosein

Martin O'Riordan

Patrick James Helland

Rico Mariani

Wilfred G Russell

Michael C Fanning

Craig Henry Wittenberg

Satish Ramchandra Thatte

Richard Douglas Hill

Doug Burger

James M Lyon

Robert Geva

Darek Josip Mihocka

Gopal Krishna R Kakivaya

Rebecca Ann Norlander

Yossi Levanoni

Eric Wendell Johnson

Nicholas Guerrera

Richard Wurdack

Mike Magruder

Matt Tolton

Scott G Robinson

Greg Hope

D T Jones

Shlomo Raikin

David T Jones

Michael Joseph Toutonghi

Amit Mital

Sean Edwin Trowbridge

Andrew Richard Putnam

Jonathan C Hawkins

Peter Lachner

Alan C Shi

Jonathan E Caves

Scott A Randell

Daniel R Spalding

Graefe Goetz

Ali-Reza Adl Tabatabai

David Callahn

Aaron Smith

Stephen F Heil

Timothy L Harris

Embodiments of systems and methods for sending messages between cores across multiple field programmable gate arrays (FPGAs) and other devices are disclosed. A uniform destination address directs a message to a core in any FPGA. Message routing within one FPGA may use a bufferless directional 2D torus Network on Chip (NOC). Message routing between FPGAs may use remote router cores coupled to the NOCs. A message from one core to another in another FPGA is routed over a NOC to a local remote router then to external remote router(s) across inter-FPGA links or networks to the remote router of the second FPGA and across a second NOC to the destination core. Messages may also be multicast to multiple cores across FPGAs. A segmented directional torus NOC is also disclosed. The insertion of shortcut routers into directional torus rings achieves shorter ring segments, reducing message delivery latency and increasing NOC bandwidth.

Composing diverse remote cores and FPGAs

Apparatus and methods are disclosed for throttling processor operation in block-based processor architectures. In one example of the disclosed technology, a block-based instruction set architecture processor includes a plurality of processing cores configured to fetch and execute a sequence of instruction blocks. Each of the processing cores includes function resources for performing operations specified by the instruction blocks. The processor further includes a core scheduler configured to allocate functional resources for performing the operations. The functional resources are allocated for executing the instruction blocks based, at least in part, on a performance metric. The performance metric can be generated dynamically or statically based on branch prediction accuracy, energy usage tolerance, and other suitable metrics.

Instruction block allocation

Apparatus and methods are disclosed for implementing block-based processors including field programmable gate-array implementations. In one example of the disclosed technology, a block-based processor includes an instruction decoder configured to generate decoded ready dependencies for a transactional block of instructions, where each of the instructions is associated with a different instruction identifier encoded in the transactional block. The processor further includes an instruction scheduler configured to issue an instruction from a set of instructions of the transactional block of instructions. The instruction is issued based on determining that decoded ready state dependencies for an instruction are satisfied. The determining includes accessing storage with the decoded ready dependencies indexed with a respective instruction identifier that is encoded in the transactional block of instructions.

Out-of-order block-based processors and instruction schedulers using ready state data indexed by instruction position identifiers

A configurable directional 2D router for Networks on Chips (NOCs) is disclosed. The router, which may be bufferless, is designed for implementation in programmable logic in FPGAs, and achieves theoretical lower bounds on FPGA resource consumption for various applications. The router employs an FPGA router switch design that consumes only one 6-LUT or 8-input ALM logic cell per router per bit of router link width. A NOC comprising a plurality of routers may be configured as a directional 2D torus, or in diverse ways, network sizes and topologies, data widths, routing functions, performance-energy tradeoffs, and other options. The router and NOC enable feasible FPGA implementation of large integrated systems on chips, interconnecting hundreds of client cores over high bandwidth links, including compute and accelerator cores, industry standard IP cores, DRAM/HBM/HMC channels, PCI Express channels, and 10G/25G/40G/100G/400G networks.

FPGA-efficient directional two-dimensional router

A fast and frugal item-state tracking scoreboard circuit is disclosed. The scoreboard maintains per-item partial states across multiple memory circuits, enabling multiple lookups per clock cycle and multiple state updates per clock cycle. In an embodiment a scoreboard is used to schedule instructions in an out-of-order processor. Each clock cycle the scoreboard indicates the busy state of an instruction's registers and may update the busy state of the destination registers of issuing instructions and completing instructions. Applications include register tracking, function-unit tracking, and cache-line state tracking, in embodiments including processor cores (including superscalar, superpipelined, and multithreaded processors), accelerators, memory systems, and networks. In an embodiment, a register-busy scoreboard circuit is implemented using FPGA LUT RAM memory. In an embodiment, a three-read/two-write per cycle register file scoreboard of 64 registers uses 16 LUTs and indicates whether an instruction is issuable in two LUT delays.

Multiported parity scoreboard circuit

Apparatus and methods are disclosed for implementing block-based processors having custom function blocks, including field-programmable gate array (FPGA) implementations. In some examples of the disclosed technology, a dynamically configurable scheduler is configured to issue at least one block-based processor instruction. A custom function block is configured to receive input operands for the instruction and generate ready state data indicating completion of a computation performed for the instruction by the respective custom function block.

Hybrid block-based processor and custom function blocks

An embodiment of a segmented directional torus network on a chip (NOC) is disclosed. The provision of shortcut routers within directional torus rings achieves shorter ring segments, reducing message delivery latency and increasing NOC bandwidth. A shortcut router may be efficiently technology mapped into fracturable FPGA lookup tables.

Shortcut routing on segmented directional torus interconnection networks

Apparatus and methods are disclosed for implementing incremental schedulers for out-of-order block-based processors, including field programmable gate array implementations. In one example of the disclosed technology, a processor includes an instruction scheduler formed by configuring one or more look up table RAMs to store ready state data for a plurality of instructions in an instruction block. The instruction scheduler further includes a plurality of queues that store ready state data for the processor and sends dependency information to ready determination logic on a first in/first out basis. The instruction scheduler selects one or more of the ready instructions to be issued and executed by the block-based processor.

Incremental scheduler for out-of-order block ISA processors

A processor core in an instruction block-based microarchitecture is configured so that an instruction window and operand buffers are decoupled for independent operation in which instructions in the block are not tied to resources such as control bits and operands that are maintained in the operand buffers. Instead, pointers are established among instructions in the block and the resources so that control state can be established for a refreshed instruction block (i.e., an instruction block that is reused without re-fetching it from an instruction cache) by following the pointers. Such decoupling of the instruction window from the operand space can provide greater processor efficiency, particularly in multiple core arrays where refreshing is utilized (for example when executing program code that uses tight loops), because the operands and control bits are pre-validated.

Decoupled processor instruction window and operand buffer

A system and method for multicast delivery of messages using a configurable directional 2D router for Networks on Chips (NOCs) is disclosed. The router is well suited for implementation in programmable logic in FPGAs and achieves theoretical lower bounds on FPGA resource consumption. A NOC comprising a plurality of routers may be configured as a directional 2D torus, or in diverse ways, network sizes and topologies, data widths, routing functions, performance-energy tradeoffs, and other options. The NOC may transmit a unicast message from one source client core to one destination client core, or a multicast message from one source client core to a plurality of destination client cores, or an arbitrary mix of unicast and multicast messages, simultaneously. A multicast message destination may include all client cores of routers with a particular first or second dimension coordinate, or all client cores, or some arbitrary subsets of client cores.

Multicast message delivery using a directional two-dimensional router and network

Composing cores and FPGAS at massive scale with directional, two dimensional routers and interconnection networks

A configurable directional 2D router for Networks on Chips (NOCs) is disclosed. The router is well suited for implementation in programmable logic in FPGAs, and achieves theoretical lower bounds on FPGA resource consumption for various applications. The router may employ an FPGA router switch design that consumes only one 6-LUT or 8-input ALM logic cell per router per bit of router link width. A NOC comprising a plurality of routers may be configured as a directional 2D torus, or in diverse ways, network sizes and topologies, data widths, routing functions, performance-energy tradeoffs, and other options. System on chip designs may employ a plurality of NOCs with different configuration parameters to customize the system to the application or workload characteristics. A great diversity of NOC client cores, for communication amongst various external interfaces and devices, and on-chip interfaces and resources, may be coupled to a router in order to efficiently communicate with other NOC client cores. The router and NOC enable feasible FPGA implementation of large integrated systems on chips, interconnecting hundreds of client cores over high bandwidth links, including compute and accelerator cores, industry standard IP cores, DRAM/HBM/HMC channels, PCI Express channels, and 10G/25G/40G/100G/400G networks.

Connecting diverse client cores using a directional two-dimensional router and network

A method including fetching a group of instructions, where the group of instructions is configured to execute atomically by a processor is provided. The method further includes decoding at least one of a first instruction or a second instruction, where: (1) decoding the first instruction results in a processing of information about a group of instructions, including information about a size of the group of instructions, and (2) decoding the second instruction results in a processing of at least one of: (a) a reference to a memory location having the information about the group of instructions, including information about the size of the group of instructions or (b) a processor status word having information about the group of instructions, including information about the size of the group of instructions.

Decoding information about a group of instructions including a size of the group of instructions

Apparatus and methods are disclosed for implementing bad jump detection in block-based processor architectures. In one example of the disclosed technology, a block-based processor includes one or more block-based processing cores configured to fetch and execute atomic blocks of instructions and a control unit configured to, based at least in part on receiving a branch signal indicating a target location is received from one of the instruction blocks, verify that the target location is a valid branch target.

Verifying branch targets

Growing community of inventors

Bellevue, WA, United States of America

Jan Stephen Gray