Nvidia Corporation

Azul Systems, Inc.

Raza Microelectronics, Inc.

Other

Jack H Choquette

Olivier Giroux

Shirish C Gadre

Xiaogang Qiu

Ronny Krashinsky

Ming Yiu Siu

Manan Patel

Omkar Paranjape

Steven James Heinrich

Michael Alan Fetterman

Robert J Stoll

Stuart F Oberman

Jonah Matthew Alben

Gil Tene

Brent Ralph Boswell

John H Edmondson

Ajay Sudarshan Tirumala

Poornachandra Rao

Alan Kaatz

Michael A Wolf

Peter Michael Nelson

Scott D Sellers

Apoorv Parle

John Erik Lindholm

Ziyad Sami Hakura

Lacky V Shah

Dale L Kirkland

Eric Brian Lum

Jeffrey A Bolz

Yury Y Uralsky

Kevin B Normoyle

Patrick Richard Brown

Robert Ohannessian

Cliff N Click, Jr

Andrew Robert Kerr

Rajballav Dash

Steve Heinrich

William P Newhall, Jr

Alexander Lev Minkin

Rajeshwaran Selvanesan

Gentaro Hirota

Nicholas Wang

Stewart Glenn Carlton

Anjana Rajendran

Eric Lyell Hill

John Tran

Manuel Olivier Gautho

Douglas J Hahn

Mark Alan Gebhart

Ravi Kiran Manyam

Ramesh Jandhyala

Jeff Tuckey

Prakash Bangalore Prabhakar

Wish Gandhi

Praveen Kaushik

Jerome Francis Duluk, Jr

Samuli Matias Laine

Brett Coon

Tero Tapani Karras

John R Nickolls

John Matthew Burgess

Gregory Muthler

Ignacio Llamas

Ronald Charles Babich, Jr

Ganesh Venkatesh

David A Kruckemyer

Gary M Tarolli

Luke Durant

Mark William Stephenson

Ivan Posva

Brian S Pharris

Daniel Dwight Grove

Ze Long

Daniel Robert Johnson

Jorge Albericio Latorre

Rishkul Kulkarni

Harold Carter Edwards

Konstantinos Kyriakopoulos

Emmett Michael Kilgariff

Vinod Grover

Sean Jeffrey Treichler

Michael C Shebanow

Rouslan Lyubomirov Dimitrov

Robert G Hathaway

Murali Sundaresan

Gregory Diamos

Stephen Jones

Gregory Scott Palmer

Charles McCarver

Jeffrey Michael Pool

Christopher Frederick Lamb

Andrei Khodakovsky

Rui Guo

Jeffrey Scott Tuckey

Richard Craig Johnson

Feng Xie

Kyrylo Perelygin

Girish Bharambe

Lei Yang

Jeff Pool

A parallel processing unit comprises a plurality of processors each being coupled to a memory access hardware circuitry. Each memory access hardware circuitry is configured to receive, from the coupled processor, a memory access request specifying a coordinate of a multidimensional data structure, wherein the memory access hardware circuit is one of a plurality of memory access circuitry each coupled to a respective one of the processors; and, in response to the memory access request, translate the coordinate of the multidimensional data structure into plural memory addresses for the multidimensional data structure and using the plural memory addresses, asynchronously transfer at least a portion of the multidimensional data structure for processing by at least the coupled processor. The memory locations may be in the shared memory of the coupled processor and/or an external memory.

Method and apparatus for efficient access to multidimensional data structures and/or other large data blocks

Apparatuses, systems, and techniques to cause information to be provided. In at least one embodiment, one or more circuits are to perform an application programming interface (API) to cause an amount of information to be accessed as a result of one or more memory transactions to be provided to one or more users.

Application programming interface to provide information

Apparatuses, systems, and techniques to adapt instructions in a SIMT architecture for execution on serial execution units. In at least one embodiment, a set of one or more threads is selected from a group of active threads associated with an instruction and the instruction is executed for the set of one or more threads on a serial execution unit.

Scalarization of instructions for SIMT architectures

A new level(s) of hierarchy—Cooperate Group Arrays (CGAs)—and an associated new hardware-based work distribution/execution model is described. A CGA is a grid of thread blocks (also referred to as cooperative thread arrays (CTAs)). CGAs provide co-scheduling, e.g., control over where CTAs are placed/executed in a processor (such as a GPU), relative to the memory required by an application and relative to each other. Hardware support for such CGAs guarantees concurrency and enables applications to see more data locality, reduced latency, and better synchronization between all the threads in tightly cooperating collections of CTAs programmably distributed across different (e.g., hierarchical) hardware domains or partitions.

Cooperative group arrays

A method, computer readable medium, and processor are disclosed for performing matrix multiply and accumulate (MMA) operations. The processor includes a datapath configured to execute the MMA operation to generate a plurality of elements of a result matrix at an output of the datapath. Each element of the result matrix is generated by calculating at least one dot product of corresponding pairs of vectors associated with matrix operands specified in an instruction for the MMA operation. A dot product operation includes the steps of: generating a plurality of partial products by multiplying each element of a first vector with a corresponding element of a second vector; aligning the plurality of partial products based on the exponents associated with each element of the first vector and each element of the second vector; and accumulating the plurality of aligned partial products into a result queue utilizing at least one adder.

Generalized acceleration of matrix multiply accumulate operations

Distributed shared memory (DSMEM) comprises blocks of memory that are distributed or scattered across a processor (such as a GPU). Threads executing on a processing core local to one memory block are able to access a memory block local to a different processing core. In one embodiment, shared access to these DSMEM allocations distributed across a collection of processing cores is implemented by communications between the processing cores. Such distributed shared memory provides very low latency memory access for processing cores located in proximity to the memory blocks, and also provides a way for more distant processing cores to also access the memory blocks in a manner and using interconnects that do not interfere with the processing cores' access to main or global memory such as hacked by an L2 cache. Such distributed shared memory supports cooperative parallelism and strong scaling across multiple processing cores by permitting data sharing and communications previously possible only within the same processing core.

Distributed shared memory

Apparatuses, systems, and techniques to perform computational operations in response to one or more compute uniform device architecture (CUDA) programs. In at least one embodiment, one or more computational operations are to cause one or more other computational operations to wait until a portion of matrix multiply-accumulate (MMA) operations have been performed.

Application programming interface to wait on matrix multiply-accumulate

This specification describes a programmatic multicast technique enabling one thread (for example, in a cooperative group array (CGA) on a GPU) to request data on behalf of one or more other threads (for example, executing on respective processor cores of the GPU). The multicast is supported by tracking circuitry that interfaces between multicast requests received from processor cores and the available memory. The multicast is designed to reduce cache (for example, layer 2 cache) bandwidth utilization enabling strong scaling and smaller tile sizes.

Programmatically controlled data multicasting across multiple compute engines

Systems and methods for an efficient and robust multiprocessor-coprocessor interface that may be used between a streaming multiprocessor and an acceleration coprocessor in a GPU are provided. According to an example implementation, in order to perform an acceleration of a particular operation using the coprocessor, the multiprocessor: issues a series of write instructions to write input data for the operation into coprocessor-accessible storage locations, issues an operation instruction to cause the coprocessor to execute the particular operation; and then issues a series of read instructions to read result data of the operation from coprocessor-accessible storage locations to multiprocessor-accessible storage locations.

Robust, efficient multiprocessor-coprocessor interface

A technique for block data transfer is disclosed that reduces data transfer and memory access overheads and significantly reduces multiprocessor activity and energy consumption. Threads executing on a multiprocessor needing data stored in global memory can request and store the needed data in on-chip shared memory, which can be accessed by the threads multiple times. The data can be loaded from global memory and stored in shared memory using an instruction which directs the data into the shared memory without storing the data in registers and/or cache memory of the multiprocessor during the data transfer.

Techniques for efficiently transferring data to a processor

Convergence of threads executing common code sections is facilitated using instructions inserted at strategic locations in computer code sections. The inserted instructions enable the threads in a warp or other group to cooperate with a thread scheduler to promote thread convergence.

Convergence among concurrently executing threads

To synchronize operations of a computing system, a new type of synchronization barrier is disclosed. In one embodiment, the disclosed synchronization barrier provides for certain synchronization mechanisms such as, for example, 'Arrive' and 'Wait' to be split to allow for greater flexibility and efficiency in coordinating synchronization. In another embodiment, the disclosed synchronization barrier allows for hardware components such as, for example, dedicated copy or direct-memory-access (DMA) engines to be synchronized with software-based threads.

High performance synchronization mechanisms for coordinating operations on a computer system

Growing community of inventors

Palo Alto, CA, United States of America

Jack H Choquette