Nvidia Corporation

Zoran Corporation

Ravi Kiran Manyam

John H Edmondson

Shirish C Gadre

Jack H Choquette

Lacky V Shah

Ronny Krashinsky

Manan Patel

Apoorv Parle

Wish Gandhi

Prakash Bangalore Prabhakar

Steve Heinrich

Jerome Francis Duluk, Jr

Olivier Giroux

Gerard K Yeh

Luke Durant

Gentaro Hirota

Brian S Pharris

Jeff Tuckey

Ze Long

Rajballav Dash

Greg Palmer

Sandeep Navada

Naman Govil

Shayani Deb

Viet Nguyen

Eric Werness

David Brian Glasco

Samuel Hammond Duncan

Patrick Rene Marchand

Alexander Lev Minkin

Wishwesh Anil Gandhi

Dane Mrazek

Yin Fung Tang

Tanmoy Mandal

Supriya Shrihari Rao

A new level(s) of hierarchy—Cooperate Group Arrays (CGAs)—and an associated new hardware-based work distribution/execution model is described. A CGA is a grid of thread blocks (also referred to as cooperative thread arrays (CTAs)). CGAs provide co-scheduling, e.g., control over where CTAs are placed/executed in a processor (such as a GPU), relative to the memory required by an application and relative to each other. Hardware support for such CGAs guarantees concurrency and enables applications to see more data locality, reduced latency, and better synchronization between all the threads in tightly cooperating collections of CTAs programmably distributed across different (e.g., hierarchical) hardware domains or partitions.

Cooperative group arrays

Distributed shared memory (DSMEM) comprises blocks of memory that are distributed or scattered across a processor (such as a GPU). Threads executing on a processing core local to one memory block are able to access a memory block local to a different processing core. In one embodiment, shared access to these DSMEM allocations distributed across a collection of processing cores is implemented by communications between the processing cores. Such distributed shared memory provides very low latency memory access for processing cores located in proximity to the memory blocks, and also provides a way for more distant processing cores to also access the memory blocks in a manner and using interconnects that do not interfere with the processing cores' access to main or global memory such as hacked by an L2 cache. Such distributed shared memory supports cooperative parallelism and strong scaling across multiple processing cores by permitting data sharing and communications previously possible only within the same processing core.

Distributed shared memory

This specification describes a programmatic multicast technique enabling one thread (for example, in a cooperative group array (CGA) on a GPU) to request data on behalf of one or more other threads (for example, executing on respective processor cores of the GPU). The multicast is supported by tracking circuitry that interfaces between multicast requests received from processor cores and the available memory. The multicast is designed to reduce cache (for example, layer 2 cache) bandwidth utilization enabling strong scaling and smaller tile sizes.

Programmatically controlled data multicasting across multiple compute engines

A method, computer readable medium, and system are disclosed for a distributed cache that provides multiple processing units with fast access to a portion of data, which is stored in local memory. The distributed cache is composed of multiple smaller caches, and each of the smaller caches is associated with at least one processing unit. In addition to a shared crossbar network through which data is transferred between processing units and the smaller caches, a dedicated connection is provided between two or more smaller caches that form a partner cache set. Transferring data through the dedicated connections reduces congestion on the shared crossbar network. Reducing congestion on the shared crossbar network increases the available bandwidth and allows the number of processing units to increase. A coherence protocol is defined for accessing data stored in the distributed cache and for transferring data between the smaller caches of a partner cache set.

Coherent caching of data for high bandwidth scaling

The invention sets forth a crossbar unit that includes multiple virtual channels, each virtual channel being a logical flow of data within the crossbar unit. Arbitration logic coupled to source client subsystems is configured to select a virtual channel for transmitting a data request or a data packet to a destination client subsystem based on the type of the source client subsystem and/or the type of data request. Higher priority traffic is transmitted over virtual channels that are configured to transmit data without causing deadlocks and/or stalls. Lower priority traffic is transmitted over virtual channels that can be stalled.

Virtual channels for effective packet transfer

A unified memory controller (UMC) is disclosed. The UMC may be used in a digital television (DTV) receiver. The UMC allows the DTV receiver to use a unified memory. The UMC accepts memory requests from various clients, and determines which requests should receive priority access to the unified memory.

Unified memory controller

Growing community of inventors

Santa Clara, CA, United States of America

Ravi Kiran Manyam