Nvidia Corporation

Mental Images Gmbh

Nvida Corporation

Alexander Georg Keller

Nikolaus Binder

Carsten Alexander Waechter

Fabrice Pierre Armand Rousselle

Jan Novák

Thomas Müller

Ken Dahm

Marc Droske

Carsten Alexander Wächter

Matthijs Jules Van Keirsbilck

Lutz Kettner

Matthias Raab

Daniel Seibert

Stefan Radig

Holger Dammertz

Samuli Matias Laine

Timo Oskari Aila

Tero Tapani Karras

Leonhard Grünschloβ

Faycal Ait Aoudia

Noah Jonathan Gamboa

Sebastian Cammerer

Dirk Gerrit Van Antwerpen

Gonçalo Filipe Torcato Mordido

Leonhard Grunschloss

Ming-Yu Liu

Jakob Hoydis

Jaakko Tapani Lehtinen

Michael Kaplan

Martin Stich

Pascal Gautron

Hendrik Lensch

Rajko Yasui-Schoeffel

Alex John Bauld Evans

Zhaoshuo Li

Chen-Hsuan Lin

Thomas Müller-Höhne

Gonçalo Felipe Torcato Mordido

Sabrina Skibak

Sabrina Dammertz

Igor Sevastianov

Johannes Hanika

Leonhard Gruenschloss

Julia Floetotto

Sehera Nawaz

Enzo Catalano

Sascha Fricke

Apparatuses, systems, and techniques to generate pixels based on other pixels. In at least one embodiment, one or more neural networks are used to generate one or more pixels based, at least in part, on sets of pixels surrounding the one or more pixels.

Neural networks to generate pixels

In photorealistic image synthesis by light transport simulation, the colors of each pixel are an integral of a high-dimensional function. However, the functions to integrate contain discontinuities that cannot be predicted efficiently. In practice, the pixel colors are estimated by using Monte Carlo and quasi-Monte Carlo methods to sample light transport paths that connect light sources and cameras and summing up the contributions to evaluate an integral. Because of the sampling, images appear noisy when the number of samples is insufficient. A low discrepancy sequence provides sample locations and these sample locations can be enumerated (assigned or distributed to pixels) according to a space-filling curve superimposed on a pixel grid. Correlations of such combinations of space-filling curves and low discrepancy sequences are analyzed, and the presented algorithms reduce correlations, are deterministic, and may be executed for each pixel in parallel.

Rendering along a space-filling curve

Artificial neural networks (ANNs) are computing systems inspired by the human brain by learning to perform tasks by considering examples. These ANNs are typically created by connecting several layers of artificial neurons using connections, where each artificial neuron is connected to every other artificial neuron either directly or indirectly to create fully connected layers within the ANN. By substituting ternary matrices for one or more fully connected layers within the ANN, a complexity and resource usage of the ANN may be reduced, while improving the performance of the ANN.

Incorporating a ternary matrix into a neural network

Monte Carlo and quasi-Monte Carlo integration are simple numerical recipes for solving complicated integration problems, such as valuating financial derivatives or synthesizing photorealistic images by light transport simulation. A drawback of a straightforward application of (quasi-)Monte Carlo integration is the relatively slow convergence rate that manifests as high error of Monte Carlo estimators. Neural control variates may be used to reduce error in parametric (quasi-)Monte Carlo integration—providing more accurate solutions in less time. A neural network system has sufficient approximation power for estimating integrals and is efficient to evaluate. The efficiency results from the use of a first neural network that infers the integral of the control variate and using normalizing flows to model a shape of the control variate.

Neural network control variates

Neural network-based structures for action user equipment device detection, estimation of time-of-arrival, and estimation of carrier frequency offset utilized with the narrowband physical random-access channel of wireless communication systems. The structure includes a neural network to generate predictions of active user equipment devices, and a twin neural network to generate time-of-arrival predictions for signals from the user equipment devices and carrier frequency offset predictions for signals from the user equipment devices.

Deep learning-based wireless communication synchronization structures

Artificial neural networks (ANNs) are computing systems that imitate a human brain by learning to perform tasks by considering examples. By representing an artificial neural network utilizing individual paths each connecting an input of the ANN to an output of the ANN, a complexity of the ANN may be reduced, and the ANN may be trained and implemented in a much faster manner when compared to an implementation using fully connected ANN graphs.

Representing a neural network utilizing paths within the network to improve a performance of the neural network

Various embodiments and implementations of graph-neural-network (GNN)-based decoding applications are disclosed. The GNN-based decoding schemes are broadly applicable to different coding schemes, and capable of operating on both binary and non-binary codewords, in different implementations. Advantageously, the inventive GNN-based decoding is scalable, even with arbitrary block lengths, and not subject to typical limits with respect to dimensionality. Decoding performance of the inventive GNN-based techniques demonstrably matches or outpaces BCH and LDPC (both regular and 5G NR) decoding algorithms, while exhibiting improvements with respect to number of iterations required and scalability of the GNN-based approach. These inventive concepts are implemented, according to various embodiments, as methods, systems, and computer program products.

Graph neural network for channel decoding

A fully-connected neural network may be configured for execution by a processor as a fully-fused neural network by limiting slow global memory accesses to reading and writing inputs to and outputs from the fully-connected neural network. The computational cost of fully-connected neural networks scale quadratically with its width, whereas its memory traffic scales linearly. Modern graphics processing units typically have much greater computational throughput compared with memory bandwidth, so that for narrow, fully-connected neural networks, the linear memory traffic is the bottleneck. The key to improving performance of the fully-connected neural network is to minimize traffic to slow 'global' memory (off-chip memory and high-level caches) and to fully utilize fast on-chip memory (low-level caches, “shared” memory, and registers), which is achieved by the fully-fused approach. A real-time neural radiance caching technique for path-traced global illumination is implemented using the fully-fused neural network for caching scattered radiance components of global illumination.

Fully-fused neural network execution

Monte Carlo and quasi-Monte Carlo integration are simple numerical recipes for solving complicated integration problems, such as valuating financial derivatives or synthesizing photorealistic images by light transport simulation. A drawback of a straightforward application of (quasi-)Monte Carlo integration is the relatively slow convergence rate that manifests as high error of Monte Carlo estimators. Neural control variates may be used to reduce error in parametric (quasi-)Monte Carlo integration&#x2014;providing more accurate solutions in less time. A neural network system has sufficient approximation power for estimating integrals and is efficient to evaluate. The efficiency results from the use of a first neural network that infers the integral of the control variate and using normalizing flows to model a shape of the control variate.

A fully-connected neural network may be configured for execution by a processor as a fully-fused neural network by limiting slow global memory accesses to reading and writing inputs to and outputs from the fully-connected neural network. The computational cost of fully-connected neural networks scale quadratically with its width, whereas its memory traffic scales linearly. Modern graphics processing units typically have much greater computational throughput compared with memory bandwidth, so that for narrow, fully-connected neural networks, the linear memory traffic is the bottleneck. The key to improving performance of the fully-connected neural network is to minimize traffic to slow 'global' memory (off-chip memory and high-level caches) and to fully utilize fast on-chip memory (low-level caches, 'shared' memory, and registers), which is achieved by the fully-fused approach. A real-time neural radiance caching technique for path-traced global illumination is implemented using the fully-fused neural network for caching scattered radiance components of global illumination.

A real-time neural radiance caching technique for path-traced global illumination is implemented using a neural network for caching scattered radiance components of global illumination. The neural (network) radiance cache handles fully dynamic scenes, and makes no assumptions about the camera, lighting, geometry, and materials. In contrast with conventional caching, the data-driven approach sidesteps many difficulties of caching algorithms, such as locating, interpolating, and updating cache points. The neural radiance cache is trained via online learning during rendering. Advantages of the neural radiance cache are noise reduction and real-time performance. Importantly, the runtime overhead and memory footprint of the neural radiance cache are stable and independent of scene complexity.

Real-time neural network radiance caching for path tracing

An apparatus, computer readable medium, and method are disclosed for decompressing compressed geometric data stored in a lossless compression format. The compressed geometric data resides within a compression block sized according to a system cache line. An indirection technique maps a global identifier value in a linear identifier space to corresponding variable rate compressed data. The apparatus may include decompression circuitry within a graphics processing unit configured to perform ray-tracing.

Block-based lossless compression of geometric data

A method, computer readable medium, and system are disclosed for performing tree traversal with backtracking in constant time. The method includes the steps of traversing a tree, maintaining a bit trail variable and a current key variable during the traversing, where the bit trail variable includes a first plurality of bits indicating tree levels on which a node has been postponed along a path from the root of the tree during the traversing, and the current key variable includes a second plurality of bits indicating a number of a current node within the tree, and performing backtracking within the tree during the traversing, utilizing the bit trail variable and the current key variable.

Tree traversal with backtracking in constant time

A method, computer readable medium, and system are disclosed for reducing noise during a rendering of a scene by sharing information that is spatially close through path space filtering. A vertex of a light transport path is selected, and one or more features of the selected vertex are quantized. A first hash is calculated based on the one or more quantized features of the selected vertex, and a collision resolution is performed within a hash table. A contribution of the light transport path at the selected vertex is accumulated to the hash table, and a counter is incremented in response to adding the contribution of the light transport path at the selected vertex to the hash table. An average contribution of the light transport path is then calculated, utilizing the counter.

Reducing noise during rendering by performing parallel path space filtering utilizing hashing

Growing community of inventors

Berlin, Germany

Alexander Georg Keller