Nvidia Corporation

Stephen Jones

Mark Hairgrove

Vivek Kini

David Fontaine

Cameron Buschardt

Lucien Dunning

Christopher Frederick Lamb

John Hubbard

Piotr Jaroszynski

Jerome Francis Duluk, Jr

Lacky V Shah

Ignacio Llamas

Philip Alexander Cuadra

Jaydeep Marathe

Daniel Elliot Wexler

Xiaohuang Huang

Jack H Choquette

Gentaro Hirota

Sean Jong Lee

Michael Murphy

Brian S Pharris

Jeff Tuckey

Bastiaan Aarts

Girish Bharambe

Rajballav Dash

Steven Arthur Gurfinkel

Ming Liang Milton Lei

Geoffrey Gerfin

Robert Overman

Kexin Yu

Sharan Chetlur

Deyu Fu

Parag Kulkarni

Techniques to modify executable graphs to perform different workloads. In at least one embodiment, an executable version of a first task graph is modified by applying a non-executable version of a second task graph to executable version of first task graph so that executable version of first task graph can perform a second workload of non-executable version of second task graph.

Techniques for modifying an executable graph to perform a workload associated with a new task graph

The present invention facilitates efficient and effective utilization of unified virtual addresses across multiple components. In one exemplary implementation, an address allocation process comprises: establishing space for managed pointers across a plurality of memories, including allocating one of the managed pointers with a first portion of memory associated with a first one of a plurality of processors; and performing a process of automatically managing accesses to the managed pointers across the plurality of processors and corresponding memories. The automated management can include ensuring consistent information associated with the managed pointers is copied from the first portion of memory to a second portion of memory associated with a second one of the plurality of processors based upon initiation of an accesses to the managed pointers from the second one of the plurality of processors.

Unified memory systems and methods

Techniques are disclosed for reducing the latency between the completion of a producer task and the launch of a consumer task dependent on the producer task. Such latency exists when the information needed to launch the consumer task is unavailable when the producer task completes. Thus, various techniques are disclosed, where a task management unit initiates the retrieval of the information needed to launch the consumer task from memory in parallel with the producer task being launched. Because the retrieval of such information is initiated in parallel with the launch of the producer task, the information is often available when the producer task completes, thus allowing for the consumer task to be launched without delay. The disclosed techniques, therefore, enable the latency between completing the producer task and launching the consumer task to be reduced.

Pre-fetching task descriptors of dependent tasks

Apparatuses, systems, and techniques for performing nested kernel execution within a parallel processing subsystem. In at least one embodiment, a parent thread launches a nested child grid on the parallel processing subsystem, and enables the parent thread to perform a thread synchronization barrier on the child grid for proper execution semantics between the parent thread and the child grid.

Technique for computational nested parallelism

Techniques are disclosed for sharing of data exchange among kernels (each a set of instructions) executing on a system having multiple processing units. In an embodiment, each processing unit includes an on-chip scratchpad memory that can be accessed by the kernels executing on the processing unit. All or a portion of the scratchpad memory can be allocated and configured, for example, such that the scratchpad is accessible to multiple kernels in parallel, to one or more kernels in serial, or a combination of both.

Persistent scratchpad memory for data exchange between programs

A system and method are provided for synchronizing threads in a divergent region of code within a multi-threaded parallel processing system. The method includes, prior to any thread entering a divergent region, generating a count that represents a number of threads that will enter the divergent region. The method also includes using the count within the divergent region to synchronize the threads in the divergent region.

System and method for synchronizing threads in a divergent region of code

A software development environment (SDE) and a method of compiling integrated source code. One embodiment of the SDE includes: (1) a parser configured to partition an integrated source code into a host code partition and a device code partition, the host code partition including a reference to a device variable, (2) a translator configured to: (2a) embed device machine code, compiled based on the device code partition, into a modified host code, (2b) define a pointer in the modified host code configured to be initialized, upon execution of the integrated source code, to a memory address allocated to the device variable, and (2c) replace the reference with a dereference to the pointer, and (3) a host compiler configured to employ a host library to compile the modified host code.

Software development environment and method of compiling integrated source code

One embodiment of the present invention sets forth a technique for dynamically allocating memory using one or more lock-free FIFOs. One or more lock-free FIFOs are populated with FIFO nodes, where each FIFO node represents a memory allocation of a predetermined size. Each particular lock-free FIFO includes memory allocations of a single size. Different lock-free FIFOs may include memory allocations for different sizes to service allocation requests for different size memory allocations. A lock-free mechanism is used to pop FIFO nodes from the FIFO. The use of the lock-free FIFO allows multiple consumers to simultaneously attempt to pop the head FIFO node without first obtaining a lock to ensure exclusive access of the FIFO.

Parallel dynamic memory allocation using a lock-free FIFO

One embodiment of the present invention sets forth a technique for performing nested kernel execution within a parallel processing subsystem. The technique involves enabling a parent thread to launch a nested child grid on the parallel processing subsystem, and enabling the parent thread to perform a thread synchronization barrier on the child grid for proper execution semantics between the parent thread and the child grid. This technique advantageously enables the parallel processing subsystem to perform a richer set of programming constructs, such as conditionally executed and nested operations and externally defined library functions without the additional complexity of CPU involvement.

Embodiments of the present invention provide a novel solution that supports the separate compilation of host code and device code used within a heterogeneous programming environment. Embodiments of the present invention are operable to link device code embedded within multiple host object files using a separate device linking operation. Embodiments of the present invention may extract device code from their respective host object files and then linked them together to form linked device code. This linked device code may then be embedded back into a host object generated by embodiments of the present invention which may then be passed to a host linker to form a host executable file. As such, device code may be split into multiple files and then linked together to form a final executable file by embodiments of the present invention.

Method and system for separate compilation of device code embedded in host code

One embodiment of the present invention sets forth a technique for dynamically allocating memory using one or more lock-free pop-only FIFOs. One or more lock-free FIFOs are populated with FIFO nodes, where each FIFO node represents a memory allocation of a predetermined size. Each particular lock-free FIFO includes memory allocations of a single size. Different lock-free FIFOs may include memory allocations for different sizes to service allocation requests for different size memory allocations. A lock-free mechanism is used to pop FIFO nodes from the FIFO. The use of the lock-free FIFO allows multiple consumers to simultaneously attempt to pop the head FIFO node without first obtaining a lock to ensure exclusive access of the FIFO.

Parallel dynamic memory allocation using a lock-free pop-only FIFO

One embodiment of the present invention sets forth a technique for dynamically allocating memory using a nested hierarchical heap. A lock-free mechanism is used to access to a hierarchical heap data structure for allocating and deallocating memory from the heap. The heap is organized as a series of levels of fixed-size blocks, where all blocks at given level are the same size. At each lower level of the hierarchy, a collection of N blocks in the lower level equals the size of a single block at the level above. When a thread requests an allocation, one or more blocks at only one level are allocated to the thread. When threads are finished using an allocation, each thread deallocates the respective allocated blocks. When all of the blocks for a level have been deallocated, defragmentation is performed at that level.

Parallel dynamic memory allocation using a nested hierarchical heap

A technique for ensuring that multiple producer threads may simultaneously write entries in a shared queue and one or more consumers may read valid data from the shared queue. Writing of the shared queue by the multiple producer threads may occur in parallel and the one or more consumer threads may read the shared queue while the producer threads write the shared queue. A 'wait-free' mechanism allows any producer thread that writes a shared queue to advance an inner pointer that is used by a consumer thread to read valid data from the shared queue. The inner pointer indicates the most recent valid entry. An output pointer is advanced with an atomic operation to indicate a next entry or portion of memory in the shared queue that is available for allocation. The shared queue may be implemented as a circular buffer.

Growing community of inventors

San Francisco, CA, United States of America

Stephen Jones