Nvidia Corporation

Intel Corporation

Wishwesh Anil Gandhi

Lacky V Shah

Jerome Francis Duluk, Jr

Timothy John Purcell

Shailendra Singh

Manas Mandal

Praveen Krishnamurthy

Balaji Vembu

Aditya Navale

Samuel Hammond Duncan

James Leroy Deming

Olivier Giroux

Gregory Scott Palmer

Alan Menezes

Praveen Joginipally

Pranav Vaidya

Feiqi Su

Michael A Parker

Karan Mehra

Jonathon Stuart Ramsey Evans

Manan Patel

Tuong P Trieu

Ram Rangan

Jonathon Stuart Ramsay Evans

Srinivas Santosh Kumar Madugula

Raghuram L

Eric Rock

Ivan Tanasic

Mark Douglas Hummel

James Michael O'Connor

Rouslan Lyubomirov Dimitrov

Alexander Lev Minkin

Aditya Sreenivas

Mark Hairgrove

Steve John Clohset

Donghyuk Lee

Raymond Hoi Man Wong

Niladrish Chatterjee

Andrei Khodakovsky

Hemayet Hossain

Debajit Bhattacharya

Kirill A Dmitriev

Sonata Gale Wen

Gaurav Uttreja

Peter B Holmquist

Suryakant Patidar

Tzyywei Hwang

Hong Jiang

Ziyad Sami Hakura

Steven E Molnar

Shirish C Gadre

Emmett Michael Kilgariff

Gregory Michael Thorson

Peter L Doyle

Thomas A Piazza

Zohar Bogin

Nirmal Raj Saxena

Patrick Richard Brown

Steven James Heinrich

Naveen Cherukuri

Gregory Muthler

Peter B Holmqvist

Alok Gupta

Trung Am Diep

George R Lynch

Ronny Krashinsky

Liang Yin

James Robertson

Vyas Venkataraman

David Fontaine

David D Lent

Arvind Gopalakrishnan

Nick Barrow-Williams

Ze Long

Ravi Kiran Manyam

Ram Gummadi

Ashish S Gadagkar

Jeffrey M Smith

Tanmoy Mandal

Geoffrey Gerfin

Dong Han Ryu

Sujoyita Kaushikkar

Vaibhav Vyas

Mathias Heyer

Terje Bergstrom

Vikram Sethi

Piyush Patel

Jeff Smith

Alexander Michael Waterman

Anup Holey

Daniel Glenn Robinson

Sami Olavi Johannes Kiminki

Supriya Shrihari Rao

Various embodiments include techniques for performing self-synchronizing remote memory operations in a data center or multiprocessor computing system. During a remote memory operation, a source processor transmits multiple data segments to a destination processor. For each data segment, the source processor transmits a remote memory operation to the destination processor that includes associated metadata that identifies the memory location of a corresponding synchronization object representing a count of data segments to be stored or a flag for each data segment to be stored. The remote memory operation along with the metadata is transmitted as a single unit to the destination processor. The destination processor splits the operation into the remote memory operation and the memory synchronization operation. As a result, the source processor avoids the need to perform a separate memory synchronization operation, thereby reducing inter-processor communications and increasing performance of remote memory operations.

Self-synchronizing remote memory operations in a data center or multiprocessor system

Apparatuses, systems, and techniques to detect memory errors and isolate or migrate partitions on a parallel processing unit using an application programming interface to facilitate parallel computing, such as CUDA. In at least one embodiment, interrupts are intercepted and processed on a graphics processing unit indicating a memory error for one or more partitions, and a policy is applied to isolate that memory error from other partitions.

Techniques for memory error isolation

A parallel processing unit (PPU), operating in a traditional processing environment or in a virtualized processing environment, can be divided into partitions. Each partition is configured to operate similarly to how the entire PPU operates. A given partition includes a subset of the computational and memory resources associated with the entire PPU. Software that executes on a CPU partitions the PPU for an admin user. A guest user is assigned to a partition and can perform processing tasks within that partition in isolation from any other guest users assigned to any other partitions. Because the PPU can be divided into isolated partitions, multiple CPU processes can efficiently utilize PPU resources.

Techniques for configuring a processor to function as multiple, separate processors in a virtualized environment

Various embodiments include techniques for performing speculative remote memory operation tracking in a multiprocessor computing system. Conventionally, transfers of data between processors and other components of a computing system require memory synchronization operations to determine that the data is valid and coherent before the data is transferred from a destination to a requesting source. Existing techniques for performing these memory synchronization operations are increasingly inefficient as the number of components in a computing system increases, particularly for remote memory operations. The disclosed techniques track remote memory operations and speculatively perform these memory synchronization operations. As a result, a given memory synchronization operation is often complete prior to the corresponding remote memory operation arrives at the destination, leading to improved efficiency and performance of remote memory operations in complex computing systems.

Speculative remote memory operation tracking for efficient memory barrier

Various embodiments include techniques for performing self-synchronizing remote memory operations in a multiprocessor computing system. During a remote memory operation in the multiprocessor computing system, a source processing unit transmits multiple segments of data to a destination processing. For each segment of data, the source processing unit transmits a remote memory operation to the destination processing unit that includes associated metadata that identifies the memory location of a corresponding synchronization object. The remote memory operation along with the metadata is transmitted as a single unit to the destination processing unit. The destination processing unit splits the operation into the remote memory operation and the memory synchronization operation. As a result, the source processing unit avoids the need to perform a separate memory synchronization operation, thereby reducing inter-processor communications and increasing performance of remote memory operations.

Self-synchronizing remote memory operations in a multiprocessor system

Various embodiments include techniques for accessing extended memory in a parallel processing system via a high-bandwidth path to extended memory residing on a central processing unit. The disclosed extended memory system extends the directly addressable high-bandwidth memory local to a parallel processing system and avoids the performance penalties associated with low-bandwidth system memory. As a result, execution threads that are highly parallelizable and access a large memory space execute with increased performance on a parallel processing system relative to prior approaches.

High bandwidth extended memory in a parallel processing system

A processing device including a first cache is coupled to a system memory and a parallel processing unit (PPU) including a second cache. An operation to modify cache lines of the second cache associated with a first aperture of the system memory is received. A first subset of cache lines of the second cache is identified. The first subset of cache lines is associated with the first aperture of the system memory and is different from a second subset of cache lines of a second aperture of the system memory. The first subset of cache lines is modified as specified by the cache operation.

Systems and methods for aperture-specific cache operations

Various embodiments include a parallel processing computer system that provides multiple memory synchronization domains in a single parallel processor to reduce unneeded synchronization operations. During execution, one execution kernel may synchronize with one or more other execution kernels by processing outstanding memory references. The parallel processor tracks memory references for each domain to each portion of local and remote memory. During synchronization, the processor synchronizes the memory references for a specific domain while refraining from synchronizing memory references for other domains. As a result, synchronization operations between kernels complete in a reduced amount of time relative to prior approaches.

Thread synchronization across memory synchronization domains

A combined on-package and off-package memory system uses a custom base-layer within which are fabricated one or more dedicated interfaces to off-package memories. An on-package processor and on-package memories are also directly coupled to the custom base-layer. The custom base-layer includes memory management logic between the processor and memories (both off and on package) to steer requests. The memories are exposed as a combined memory space having greater bandwidth and capacity compared with either the off-package memories or the on-package memories alone. The memory management logic services requests while maintaining quality of service (QoS) to satisfy bandwidth requirements for each allocation. An allocation may include any combination of the on and/or off package memories. The memory management logic also manages data migration between the on and off package memories.

Combined on-package and off-package memory system

A parallel processing unit (PPU) can be divided into partitions. Each partition is configured to operate similarly to how the entire PPU operates. A given partition includes a subset of the computational and memory resources associated with the entire PPU. Software that executes on a CPU partitions the PPU for an admin user. A guest user is assigned to a partition and can perform processing tasks within that partition in isolation from any other guest users assigned to any other partitions. Because the PPU can be divided into isolated partitions, multiple CPU processes can efficiently utilize PPU resources.

Techniques for configuring a processor to function as multiple, separate processors

Techniques for reconfiguring partitions in a parallel processing system

Accesses between a processor and its external memory is reduced when the processor internally maintains a compressed version of values stored in the external memory. The processor can then refer to the compressed version rather than access the external memory. One compression technique involves maintaining a dictionary on the processor mapping portions of a memory to values. When all of the values of a portion of memory are uniform (e.g., the same), the value is stored in the dictionary for that portion of memory. Thereafter, when the processor needs to access that portion of memory, the value is retrieved from the dictionary rather than from external memory. Techniques are disclosed herein to extend, for example, the capabilities of such dictionary-based compression so that the amount of accesses between the processor and its external memory are further reduced.

Techniques for dynamically compressing memory regions having a uniform value

Growing community of inventors

Sunnyvale, CA, United States of America

Wishwesh Anil Gandhi