Nvidia Corporation

Sony Corporation

Sony Electronics Inc

Other

Sony Electronics Inc., Jointy

Shirish C Gadre

Jack H Choquette

Steven James Heinrich

Michael Alan Fetterman

Taner E Ozcelik

Omkar Paranjape

Lacky V Shah

Olivier Giroux

Manan Patel

Ronny Krashinsky

Xiaogang Qiu

John H Edmondson

Ashish Karandikar

Robert Ohannessian

Rajeshwaran Selvanesan

Luke Durant

Nicholas Wang

Stephen Donald Lew

Anjana Rajendran

Poornachandra Rao

Subroto Bose

Philip Alexander Cuadra

Stewart Glenn Carlton

Ajay Sudarshan Tirumala

Eric Lyell Hill

Charles McCarver

Gerald F Luiz

Ramesh Jandhyala

Mark Alan Gebhart

Alan Kaatz

Gentaro Hirota

Franciscus W Sijstermans

Douglas J Hahn

Moshe Bublil

Arthur Merlin Danskin

Steve Heinrich

Nirmal Raj Saxena

Alexander Lev Minkin

Brian Matthew Fahs

Bryon S Nordquist

Eric T Anderson

Joel James McCormack

Stefano A Pescador

Andrew Robert Kerr

Elif Albuz

Ze Long

Ravi Kiran Manyam

Nick Barrow-Williams

Prakash Bangalore Prabhakar

Wish Gandhi

Apoorv Parle

Maciej Piotr Tyrlik

Gong-san Yu

Rajballav Dash

Michael Siu

Jerome Francis Duluk, Jr

Henry Packard Moreton

Eric Brian Lum

Yury Y Uralsky

David Conrad Tannenbaum

Christopher Thomas Cheng

Brian S Pharris

Jonathan J Dunaisky

John Tran

Jeff Tuckey

Edward J Paluch

Tomonari Tohara

Praveen Kaushik

Fang-Chuan Wu

Greg Palmer

Sandeep Navada

Syed Reza

Ikuo Tsukagoshi

Stuart F Oberman

Emmett Michael Kilgariff

John Matthew Burgess

Ming Yiu Siu

Rouslan Lyubomirov Dimitrov

Wishwesh Anil Gandhi

Michael A Parker

Christian Rouet

Daniel P Wilde

Frederick R Gruner

Jean Swey Kao

Robert Jan Schutten

James Robertson

Robert J Stoll

Liang Yin

Gregory Scott Palmer

Martin Stich

David Fontaine

Andrei Khodakovsky

Jeffrey Scott Tuckey

Lei Yang

Zhiqiang Jonathan Su

Pattabiraman Subramanian

Yew-Koon Tan

Hemayet Hossain

Scott Ricketts

Frank Joseph Eaton

A parallel processing unit comprises a plurality of processors each being coupled to a memory access hardware circuitry. Each memory access hardware circuitry is configured to receive, from the coupled processor, a memory access request specifying a coordinate of a multidimensional data structure, wherein the memory access hardware circuit is one of a plurality of memory access circuitry each coupled to a respective one of the processors; and, in response to the memory access request, translate the coordinate of the multidimensional data structure into plural memory addresses for the multidimensional data structure and using the plural memory addresses, asynchronously transfer at least a portion of the multidimensional data structure for processing by at least the coupled processor. The memory locations may be in the shared memory of the coupled processor and/or an external memory.

Method and apparatus for efficient access to multidimensional data structures and/or other large data blocks

Various embodiments include a parallel processing computer system that provides multiple memory synchronization domains in a single parallel processor to reduce unneeded synchronization operations. During execution, one execution kernel may synchronize with one or more other execution kernels by processing outstanding memory references. The parallel processor tracks memory references for each domain to each portion of local and remote memory. During synchronization, the processor synchronizes the memory references for a specific domain while refraining from synchronizing memory references for other domains. As a result, synchronization operations between kernels complete in a reduced amount of time relative to prior approaches.

Thread synchronization across memory synchronization domains

A new level(s) of hierarchy—Cooperate Group Arrays (CGAs)—and an associated new hardware-based work distribution/execution model is described. A CGA is a grid of thread blocks (also referred to as cooperative thread arrays (CTAs)). CGAs provide co-scheduling, e.g., control over where CTAs are placed/executed in a processor (such as a GPU), relative to the memory required by an application and relative to each other. Hardware support for such CGAs guarantees concurrency and enables applications to see more data locality, reduced latency, and better synchronization between all the threads in tightly cooperating collections of CTAs programmably distributed across different (e.g., hierarchical) hardware domains or partitions.

Cooperative group arrays

Various embodiments include techniques for managing cache memory in a computing system. The computing system includes a sectored cache memory that provides a mechanism for software applications to directly invalidate data items stored in the cache memory on a sector-by-sector basis, where a sector is smaller than a cache line. When all sectors in a cache line have been invalidated, the cache line is implicitly invalidated, freeing the cache line to be reallocated for other purposes. In cases where the data items to be invalidated can be aligned to sector boundaries, the disclosed techniques effectively use status indicators in the cache tag memory to track which sectors, and corresponding data items, have been invalidated by the software application. Thus, the disclosed techniques thereby enable a low-overhead solution for invalidating individual data items that are smaller than a cache line without additional tracking data structures or consuming additional memory transfer bandwidth.

Cache memory with per-sector cache residency controls

Distributed shared memory (DSMEM) comprises blocks of memory that are distributed or scattered across a processor (such as a GPU). Threads executing on a processing core local to one memory block are able to access a memory block local to a different processing core. In one embodiment, shared access to these DSMEM allocations distributed across a collection of processing cores is implemented by communications between the processing cores. Such distributed shared memory provides very low latency memory access for processing cores located in proximity to the memory blocks, and also provides a way for more distant processing cores to also access the memory blocks in a manner and using interconnects that do not interfere with the processing cores' access to main or global memory such as hacked by an L2 cache. Such distributed shared memory supports cooperative parallelism and strong scaling across multiple processing cores by permitting data sharing and communications previously possible only within the same processing core.

Distributed shared memory

Various techniques for accelerating dynamic programming algorithms are provided. For example, a fused addition and comparison instruction, a three-operand comparison instruction, and a two-operand comparison instruction are used to accelerate a Needleman-Wunsch algorithm that determines an optimized global alignment of subsequences over two entire sequences. In another example, the fused addition and comparison instruction is used in an innermost loop of a Floyd-Warshall algorithm to reduce the number of instructions required to determine shortest paths between pairs of vertices in a graph. In another example, a two-way single instruction multiple data (SIMD) floating point variant of the three-operand comparison instruction is used to reduce the number of instructions required to determine the median of an array of floating point values.

Implementing specialized instructions for accelerating dynamic programming algorithms

This specification describes a programmatic multicast technique enabling one thread (for example, in a cooperative group array (CGA) on a GPU) to request data on behalf of one or more other threads (for example, executing on respective processor cores of the GPU). The multicast is supported by tracking circuitry that interfaces between multicast requests received from processor cores and the available memory. The multicast is designed to reduce cache (for example, layer 2 cache) bandwidth utilization enabling strong scaling and smaller tile sizes.

Programmatically controlled data multicasting across multiple compute engines

Various embodiments include a system for managing cache memory in a computing system. The system includes a sectored cache memory that provides a mechanism for sharing sectors in a cache line among multiple cache line allocations. Traditionally, different cache line allocations are assigned to different cache lines in the cache memory. Further, cache line allocations may not use all of the sectors of the cache line, leading to low utilization of the cache memory. With the present techniques, multiple cache lines share the same cache line, leading to improved cache memory utilization relative to prior techniques. Further, sectors of cache allocations can be assigned to reduce data bank conflicts when accessing cache memory. Reducing such data bank conflicts can result in improved memory access performance, even when cache lines are shared with multiple allocations.

Hybrid allocation of data lines in a streaming cache memory

A technique for block data transfer is disclosed that reduces data transfer and memory access overheads and significantly reduces multiprocessor activity and energy consumption. Threads executing on a multiprocessor needing data stored in global memory can request and store the needed data in on-chip shared memory, which can be accessed by the threads multiple times. The data can be loaded from global memory and stored in shared memory using an instruction which directs the data into the shared memory without storing the data in registers and/or cache memory of the multiprocessor during the data transfer.

Techniques for efficiently transferring data to a processor

Various techniques for accelerating Smith-Waterman sequence alignments are provided. For example, threads in a group of threads are employed to use an interleaved cell layout to store relevant data in registers while computing sub-alignment data for one or more local alignment problems. In another example, specialized instructions that reduce the number of cycles required to compute each sub-alignment score are utilized. In another example, threads are employed to compute sub-alignment data for a subset of columns of one or more local alignment problems while other threads begin computing sub-alignment data based on partial result data received from the preceding threads. After computing a maximum sub-alignment score, a thread stores the maximum sub-alignment score and the corresponding position in global memory.

Techniques for storing sub-alignment data when accelerating Smith-Waterman sequence alignments

To synchronize operations of a computing system, a new type of synchronization barrier is disclosed. In one embodiment, the disclosed synchronization barrier provides for certain synchronization mechanisms such as, for example, 'Arrive' and 'Wait' to be split to allow for greater flexibility and efficiency in coordinating synchronization. In another embodiment, the disclosed synchronization barrier allows for hardware components such as, for example, dedicated copy or direct-memory-access (DMA) engines to be synchronized with software-based threads.

High performance synchronization mechanisms for coordinating operations on a computer system

In a streaming cache, multiple, dynamically sized tracking queues are employed. Request tracking information is distributed among the plural tracking queues to selectively enable out-of-order memory request returns. A dynamically controlled policy assigns pending requests to tracking queues, providing for example in-order memory returns in some contexts and/or for some traffic and out of order memory returns in other contexts and/or for other traffic.

Out of order memory request tracking structure and technique

Implementing specialized instructions for accelerating Smith-Waterman sequence alignments

Growing community of inventors

San Jose, CA, United States of America

Shirish C Gadre