Microsoft Technology Licensing, LLC

Xilinx, Inc.

University of Washington

Andrew Richard Putnam

Douglas Christopher Burger

Stephen Francis Heil

Eric S Chung

Adrian Michael Caulfield

Sitaram V Lanka

Derek Chiou

Yi Xiao

Daniel Max Firestone

Sambhrama Madhusudhan Mundkur

Somesh Chaturmohta

Norman C Lam

Fengfen Liu

Michael David Haselman

Prasanna Sundararajan

Jan Stephen Gray

James R Larus

Aaron L Smith

David W Bennett

Jeffrey M Mason

Mark Henry Oskin

Susan Jane Eggers

Alec Kochevar-Cureton

Andrew Michalski Schwerin

Jack Thomas Lavier

Steven James Swanson

Martha Allen Mercaldi

Kenneth Alan Michelson

Andrew Kinoshita Petersen

Aaron Smith

Stephen F Heil

Derek T Chiou

A physical server with an offload card including a SoC (system-on-chip) and a FPGA (field programmable gate array) is disclosed. According to one set of embodiments, the SoC can be configured to offload one or more hypervisor functions from a CPU complex of the server that are suited for execution in software, and the FPGA can be configured to offload one or more hypervisor functions from the CPU complex that are suited for execution in hardware.

Server offload card with SoC and FPGA

A data processing system is described herein that includes two or more software-driven host components. The two or more host components collectively provide a software plane. The data processing system also includes two or more hardware acceleration components (such as FPGA devices) that collectively provide a hardware acceleration plane. A common physical network allows the host components to communicate with each other, and which also allows the hardware acceleration components to communicate with each other. Further, the hardware acceleration components in the hardware acceleration plane include functionality that enables them to communicate with each other in a transparent manner without assistance from the software plane.

Data processing system having a hardware acceleration plane and a software plane

Aspects extend to methods, systems, and computer program products for partially reconfiguring acceleration components. Partial reconfiguration can be implemented for any of a variety of reasons, including to address an error in functionality at the acceleration component or to update functionality at the acceleration component. During partial reconfiguration, connectivity can be maintained for any other functionality at the acceleration component untouched by the partial reconfiguration. Partial reconfiguration is more efficient to deploy than full reconfiguration of an acceleration component.

Partially reconfiguring acceleration components

Distributed computing systems, devices, and associated methods of packet routing are disclosed herein. In one embodiment, a method includes receiving, from a computing network, a packet at a packet processor of a server. The method also includes matching the received packet with a flow in a flow table contained in the packet processor and determining whether the action indicates that the received packet is to be forwarded to a NIC buffer in the outbound processing path of the packet processor instead of the NIC. The method further includes in response to determining that the action indicates that the received packet is to be forwarded to the NIC buffer, forwarding the received packet to the NIC buffer and processing the packet in the NIC buffer to forward the packet to the computer network without exposing the packet to the main processor.

Network traffic routing in distributed computing systems

Aspects extend to methods, systems, and computer program products for allocating acceleration component functionality for supporting services. A service manager uses a finite number of acceleration components to accelerate services. Acceleration components can be allocated in a manner that balances load in a hardware acceleration plane, minimizes role switching, and adapts to demand changes. When role switching is appropriate, less extensive mechanisms (e.g., based on configuration data versus image files) can be used to switch roles to the extent possible.

Allocating acceleration component functionality for supporting services

Distributed computing systems, devices, and associated methods of packet routing are disclosed herein. In one embodiment, a computing device includes a field programmable gate array ('FPGA') that includes an inbound processing path and outbound processing path in opposite processing directions. The inbound processing path can forward a packet received from the computer network to a buffer on the FPGA instead of the NIC. The outbound processing path includes an outbound multiplexer having a rate limiter circuit that only forwards the received packet from the buffer back to the computer network when a virtual port corresponding to the packet has sufficient transmission allowance. The outbound multiplexer can also periodically increment the transmission allowance based on a target bandwidth for the virtual port.

Network traffic rate limiting in computing systems

A hardware acceleration component is provided that includes a plurality of hardware clusters, each hardware cluster comprising a plurality of soft processor cores and a functional circuit. The plurality of soft processor cores share the functional circuit.

Free form expression accelerator with thread length-based thread assignment to clustered soft processor cores that share a functional circuit

A method is provided for processing on an acceleration component a deep neural network. The method includes configuring the acceleration component to perform forward propagation and backpropagation stages of the deep neural network. The acceleration component includes an acceleration component die and a memory stack disposed in an integrated circuit package. The memory stack has a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW.

Deep neural network processing on hardware accelerators with stacked memory

Dynamic power routing is utilized to route power from other components, which are transitioned to lower power consuming states, in order to accommodate more efficient processing of computational tasks by hardware accelerators, thereby staying within electrical power thresholds that would otherwise not have accommodated simultaneous full-power operation of the other components and such hardware accelerators. Once a portion of a workflow is being processed by hardware accelerators, the workflow, or the hardware accelerators, can be self-throttling to stay within power thresholds, or they can be throttled by independent coordinators, including device-centric and system-wide coordinators. Additionally, predictive mechanisms can be utilized to obtain available power in advance, by proactively transitioning other components to reduced power consuming states, or reactive mechanisms can be utilized to only transition components to reduced power consuming states when a specific need for increased hardware accelerator power is identified.

Dynamic power routing to hardware accelerators

Aspects extend to methods, systems, and computer program products for changing between different roles at acceleration components. Changing roles at an acceleration component can be facilitated without loading an image file to configure or partially reconfigure the acceleration component. At configuration time, an acceleration component can be configured with a framework and a plurality of selectable roles. The framework also provides a mechanism for loading different selectable roles for execution at the acceleration component (e.g., the framework can include a superset of instructions for providing any of a plurality of different roles). The framework can receive requests for specified roles from other components and switch to a subset of instructions for the specified roles. Switching between subsets of instructions at an acceleration component is a lower overhead operation relative to reconfiguring or partially reconfiguring an acceleration component by loading an image file.

Changing between different roles at acceleration components

A method is provided for processing on an acceleration component a machine learning classification model. The machine learning classification model includes a plurality of decision trees, the decision trees including a first amount of decision tree data. The acceleration component includes an acceleration component die and a memory stack disposed in an integrated circuit package. The memory die includes an acceleration component memory having a second amount of memory less than the first amount of decision tree data. The memory stack includes a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW. The method includes slicing the model into a plurality of model slices, each of the model slices having a third amount of decision tree data less than or equal to the second amount of memory, storing the plurality of model slices on the memory stack, and for each of the model slices, copying the model slice to the acceleration component memory, and processing the model slice using a set of input data on the acceleration component to produce a slice result.

Machine learning classification on hardware accelerators with stacked memory

A decision tree multi-processor system includes a plurality of decision tree processors that access a common feature vector and execute one or more decision trees with respect to the common feature vector. A related method includes providing a common feature vector to a plurality of decision tree processors implemented within an on-chip decision tree scoring system, and executing, by the plurality of decision tree processors, a plurality off decision trees, by reference to the common feature vector. A related decision tree-walking system includes feature storage that stores a common feature vector and a plurality of decision tree processors that access the common feature vector from the feature storage and execute a plurality of decision trees by comparing threshold values of the decision trees to feature values within the common feature vector.

Parallel decision tree processor architecture

A data processing system is described herein that includes two or more software-driven host components that collectively provide a software plane. The data processing system further includes two or more hardware acceleration components that collectively provide a hardware acceleration plane. The hardware acceleration plane implements one or more services, including at least one multi-component service. The multi-component service has plural parts, and is implemented on a collection of two or more hardware acceleration components, where each hardware acceleration component in the collection implements a corresponding part of the multi-component service. Each hardware acceleration component in the collection is configured to interact with other hardware acceleration components in the collection without involvement from any host component. A function parsing component is also described herein that determines a manner of parsing a function into the plural parts of the multi-component service.

Growing community of inventors

Seattle, WA, United States of America

Andrew Richard Putnam