Advanced Micro Devices Corporation

Microsoft Technology Licensing, LLC

Intel Corporation

Hewlett-packard Development Company, L.p.

Steven Karl Reinhardt

Shubhendu Sekhar Mukherjee

Bradford Michael Beckmann

Joel S Emer

Stephen Sangho Youn

Jeremy Halden Fowers

Kalin Ovtcharov

Marc S Orr

Christopher T Weaver

David Allen Wood

Eric S Chung

Hui Geng

Lok Chand Koppaka

Ming Gang Liu

Benedict R Gaster

Michael Wayne LeBeane

Michael J Smith

Mark Donald Hill

Mithuna Shamabhat Thottethodi

Arkaprava Basu

Shuai Che

Michael L Chu

Walter B Benton

Blake A Hechtman

Dan Zhang

Derek R Hower

Vinod Tipparaju

Eric Sen Chung

Abhisek Pan

Nilay Vaish

Tushar Krishna

Bradfod M Beckmann

Joseph Anthony Mayer, Ii

Michael W Lebeane

The present disclosure relates to devices for using a configurable stacked architecture for a fixed function datapath with an accelerator for accelerating an operation or a layer of a deep neural network (DNN). The stacked architecture may have a fixed function datapath that includes one or more configurable micro-execution units that execute a series of vector, scalar, reduction, broadcasting, and normalization operations for a DNN layer operation. The fixed function datapath may be customizable based on the DNN or the operation.

Semi-programmable and reconfigurable co-accelerator for a deep neural network with normalization or non-linearity

The present disclosure relates to devices and methods for using a banked memory structure with accelerators. The devices and methods may segment and isolate dataflows in datapath and memory of the accelerator. The devices and methods may provide each data channel with its own register memory bank. The devices and methods may use a memory address decoder to place the local variables in the proper memory bank.

Banked memory architecture for multiple parallel datapath channels in an accelerator

Deep neural network accelerators (DNNs) with independent datapaths for simultaneous processing of different classes of operations and related methods are described. An example DNN accelerator includes an instruction dispatcher for receiving chains of instructions having both instructions for performing a first class of operations and a second class of operations corresponding to a neural network model. The DNN accelerator further includes a first datapath and a second datapath, where each is configured to execute at least one instruction chain locally before outputting any results. The instruction dispatcher is configured to forward instructions for performing the first class of operations to the first datapath and forward instructions for performing the second class of operations to the second datapath to overlap in time a performance of at least a subset of the first class of operations with a performance of at least a subset of the second class of operations.

Deep neural network accelerator with independent datapaths for simultaneous processing of different classes of operations

A computing system is described herein that expedites deep neural network (DNN) operations or other processing operations using a hardware accelerator. The hardware accelerator, in turn, includes a tensor-processing engine that works in conjunction with a scalar-processing unit (SPU). The tensor-processing engine handles various kinds of tensor-based operations required by the DNN, such as multiplying vectors by matrices, combining vectors with other vectors, transforming individual vectors, etc. The SPU performs scalar-based operations, such as forming the reciprocal of a scalar, generating the square root of a scalar, etc. According to one illustrative implementation, the computing system uses the same vector-based programmatic interface to interact with both the tensor-processing engine and the SPU.

Tensor-based hardware accelerator including a scalar-processing unit

Techniques for scheduling processing tasks in a device having multiple computing elements are disclosed. A network interface controller of the device receives processing tasks, for execution on the computing elements, from a network that is external to the device. The network interface controller schedules the tasks for execution on the computing devices based on policy data available to the network interface controller. A scheduler within the network interface controller, which can be implemented as a standalone processing unit (such as a microcontroller, a programmable processing core, or an application specific integrated circuit), performs such scheduling, thereby freeing the central processing unit of the device from the burden of performing scheduling operations. The scheduler schedules the tasks according to any technically feasible scheduling technique.

Network interface controller-based scheduling of processing tasks in a distributed computing system

Methods, devices, and systems for transmitting data over a computer communications network are disclosed. A queue of communications commands can be pre-generated using a central processing unit (CPU) and stored in a device memory of a network interface controller (NIC). Thereafter, if a graphics processing unit (GPU) has data to communicate to a remote GPU, it can store the data in a send buffer, where the location in the buffer is pointed to by a pre-generated command. The GPU can then signal to the interface device that the data is ready, triggering execution of the pre-generated command to send the data.

GPU remote communication with triggered operations

A hardware accelerator having an efficient instruction set is disclosed. An apparatus may comprise logic configured to access a first and a second machine instruction. The second machine instruction may be missing a tensor operand needed to execute the second machine instruction. The logic may be further configured to execute the first machine instruction, resulting in a tensor. The logic may be further configured to execute the second machine instruction using the resultant tensor as the missing tensor operand.

Tensor processor instruction set architecture

A processor comprising hardware logic configured to execute of a first wavefront in a hardware resource and stop execution of the first wavefront before the first wavefront completes. The processor schedules a second wavefront for execution in the hardware resource.

Wavefront resource virtualization

Tensor register files in a hardware accelerator are disclosed. An apparatus may comprise tensor operation calculators each configured to perform a type of tensor operation. The apparatus may also comprises tensor register files, each of which is associated with one of the tensor operation calculators. The apparatus may also comprises logic configured to store respective ones of the tensors in the plurality of tensor register files in accordance with the type of tensor operation to be performed on the respective tensors. The apparatus may also control read access to tensor register files based on a type of tensor operation that a machine instruction is to perform.

Tensor register files

A processor circuit is provided that includes an input terminal and an output terminal, a plurality of vector processor operation circuits, a selector circuit coupled to the input terminal, the output terminal, and each of the vector processor operation circuits, and a scheduler circuit adapted to control the selector circuit to configure a vector processing pipeline comprising zero, one or more of the vector processor operation circuits in any order between the input terminal and the output terminal.

Multifunction vector processor circuits

A system and method for efficient management of network traffic management of highly data parallel computing. A processing node includes one or more processors capable of generating network messages. A network interface is used to receive and send network messages across a network. The processing node reduces at least one of a number or a storage size of the original network messages into one or more new network messages. The new network messages are sent to the network interface to send across the network.

Message aggregation, combining and compression for efficient data communications in GPU-based clusters

A conditional fetch-and-phi operation tests a memory location to determine if the memory locations stores a specified value and, if so, modifies the value at the memory location. The conditional fetch-and-phi operation can be implemented so that it can be concurrently executed by a plurality of concurrently executing threads, such as the threads of wavefront at a GPU. To execute the conditional fetch-and-phi operation, one of the concurrently executing threads is selected to execute a compare-and-swap (CAS) operation at the memory location, while the other threads await the results. The CAS operation tests the value at the memory location and, if the CAS operation is successful, the value is passed to each of the concurrently executing threads.

Conditional atomic operations in single instruction multiple data processors

Growing community of inventors

Vancouver, WA, United States of America

Steven Karl Reinhardt