Intel Corporation

Nvidia Corporation

Marco Salvi

Aaron Eliot Lefohn

Karthik Vaidyanathan

Anjul Patney

Carl Jacob Munkberg

Robert M Toth

Christopher Ryan Wyman

Henry Packard Moreton

Yury Y Uralsky

Franz Petrik Clarberg

Jon Niklas Theodor Hasselgren

Tomas G Akenine-Moller

David Patrick Luebke

Jon N Hasselgren

Gabor Liktor

John Matthew Burgess

Anton S Kaplanyan

Donald Lee Brittain

Samuli Matias Laine

Eric Brian Lum

Tomas Guy Akenine-Möller

Gregory Muthler

Magnus Andersson

Andrew T Lauritzen

Rahul P Sathe

Kayvon Fatahalian

Eric Enderton

Craig E Kolb

Christoph Kubisch

Yong He

G Evan Hart, Jr

Nikhil Dixit

Yubo Zhang

Alessandro Dal Corso

Timo Oskari Aila

Larry D Seiler

Jim K Nilsson

Tomasz Janczak

Jaakko Tapani Lehtinen

Jacopo Pantaleoni

Josef Bo Spjut

Joohwan Kim

Johan Pontus Andersson

Kiril Vidimce

Pascal Gautron

Nir Benty

Brennan Shacklett

Chakravarty Reddy Alla Chaitanya

Christoph H Schied

Matt Pharr

Jefferson D Montgomery

Tzu-Mao Li

A Displaced Micro-mesh (DMM) primitive enables high complexity geometry for ray and path tracing while minimizing the associated builder costs and preserving high efficiency. A structured, hierarchical representation implicitly encodes vertex positions of a triangle micro-mesh based on a barycentric grid, and enables microvertex displacements to be encoded efficiently (e.g., as scalars linearly interpolated between minimum and maximum triangle surfaces). The resulting displaced micro-mesh primitive provides a highly compressed representation of a potentially vast number of displaced microtriangles that can be stored in a small amount of space. Improvements in ray tracing hardware permit automatic processing of such primitive for ray-geometry intersection testing by ray tracing circuits without requiring intermediate reporting to a shader.

Displaced micro-meshes for ray and path tracing

Apparatuses, systems, and techniques are presented to generate images. In at least one embodiment, at least a first optical flow network (OFN) and at least a first reconstruction network (RN) can be used to generate one or more images based, at least in part, upon the OFN and the RN using a shared loss function.

Image generation using one or more neural networks

This application discloses techniques for generating and querying projective hash maps. More specifically, projective hash maps can be used for spatial hashing of data related to N-dimensional points. Each point is projected onto a projection surface to convert the three-dimensional (3D) coordinates for the point to two-dimensional (2D) coordinates associated with the projection surface. Hash values based on the 2D coordinates are then used as an index to store data in the projective hash map. Utilizing the 2D coordinates rather than the 3D coordinates allows for more efficient searches to be performed to locate points in the 3D space. In particular, projective hash maps can be utilized by graphics applications for generating images, and the improved efficiency can, for example, enable a game streaming application on a server to render images transmitted to a user device via a network at faster frame rates.

Projective hash maps

In some embodiments, a given frame or picture may have different shading rates. In one embodiment in some areas of the frame or picture the shading rate may be less than once per pixel and in other places it may be once per pixel. Examples where the shading rate may be reduced include areas where there is motion and camera defocus, areas of peripheral blur, and in general, any case where the visibility is reduced anyway. The shading rate may be changed in a region, such as a shading quad, by changing the size of the region.

Decoupled shading pipeline

A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.

Neural network system with temporal feedback for denoising of rendered sequences

Neural network system with temporal feedback for adaptive sampling and denoising of rendered sequences

A method dynamically selects one of a first sampling order and a second sampling order for a ray trace of pixels in a tile where the selection is based on a motion vector for the tile. The sampling order may be a bowtie pattern or an hourglass pattern. Subframes generated based on the sampling order are communicated over a bus along with motion vectors for tiles of the subframes.

Reconstruction for temporally dense ray trace rendering

A method, computer readable medium, and system are disclosed for performing spatiotemporal filtering. The method includes identifying image data to be rendered, reconstructing the image data to create reconstructed image data, utilizing a filter including a neural network having one or more skip connections and one or more recurrent layers, and returning the reconstructed image data.

Filtering image data using a neural network

A neural network structure, namely a warped external recurrent neural network, is disclosed for reconstructing images with synthesized effects. The effects can include motion blur, depth of field reconstruction (e.g., simulating lens effects), and/or anti-aliasing (e.g., removing artifacts caused by sampling frequency). The warped external recurrent neural network is not recurrent at each layer inside the neural network. Instead, the external state output by the final layer of the neural network is warped and provided as a portion of the input to the neural network for the next image in a sequence of images. In contrast, in a conventional recurrent neural network, hidden state generated at each layer is provided as a feedback input to the generating layer. The neural network can be implemented, at least in part, on a processor. In an embodiment, the neural network is implemented on at least one parallel processing unit.

Motion blur and depth of field reconstruction through temporally stable neural networks

A method, computer readable medium, and system are disclosed for performing spatiotemporal filtering. The method includes the steps of applying, utilizing a processor, a temporal filter of a filtering pipeline to a current image frame, using a temporal reprojection, to obtain a color and auxiliary information for each pixel within the current image frame, providing the auxiliary information for each pixel within the current image frame to one or more subsequent filters of the filtering pipeline, and creating a reconstructed image for the current image frame, utilizing the one or more subsequent filters of the filtering pipeline.

Performing spatiotemporal filtering

A method, computer readable medium, and system are disclosed for performing stable ray tracing. The method includes the steps of identifying a plurality of old hit points used in a previously rendered frame, re-projecting the plurality of old hit points within a current frame to create a plurality of samples within a screen space of the current frame, adjusting the plurality of samples within the screen space of the current frame, based on one or more criteria, for each of the plurality of samples, tracing a ray from the sample toward a corresponding old hit point for the sample to determine a current hit point corresponding to the sample for the current frame, where the current hit point may include the corresponding old hit point for the sample or an updated hit point for the sample, shading at least a portion of the plurality of current hit points to obtain a color for each of the plurality of samples within the screen space of the current frame, and reconstructing a final color for a plurality of pixels in the screen space of the current frame, utilizing the color for each of the plurality of samples within the screen space of the current frame.

Stable ray tracing

A system, method, and computer readable medium for inverse graphics rendering comprise a differentiable rendering pipeline and a gradient descent optimization engine. A given scene is described using scene parameters. Visibility functions, and other rendered functions, are constructed to be continuous and differentiable, allowing the optimization engine and the rendering pipeline to efficiently iterate through increasingly refined scene models.

Differentiable rendering pipeline for inverse graphics

Growing community of inventors

San Francisco, CA, United States of America

Marco Salvi