Nvidia Corporation

Morgan McGuire

David Patrick Luebke

Peter Schuyler Shirley

Christopher Ryan Wyman

Michael Mara

Aaron Eliot Lefohn

Michael Stengel

Peter-Pike Sloan

Brent Leonard Oster

Anjul Patney

Josef Bo Spjut

Qi Sun

Ben Boudaoud

Cyril Crassin

Omer Shapira

Samuli Matias Laine

Rahul P Sathe

Shalini De Mello

Holger Heinrich Gruen

Joohwan Kim

Alexander Majercik

Cyril Jean-Francois Crassin

Adam Christopher Marrs

Zander Majercik

Ziyad Sami Hakura

Timo Oskari Aila

Emmett Michael Kilgariff

Yury Y Uralsky

Jacopo Pantaleoni

Jonathan Cohen

Dawid Stanislaw Pajak

Jonghyun Kim

Ward Lopes

Matthew Milton Pharr

Eric Enderton

Kaan Aksit

Thomas Müller

Pete Shirley

Pyarelal Knowles

Kamran Binaee

Harish Vutukuru

David Augustus Hart

One embodiment of a method includes calculating one or more activation values of one or more neural networks trained to infer eye gaze information based, at least in part, on eye position of one or more images of one or more faces indicated by an infrared light reflection from the one or more images.

Synthetic infrared image generation for machine learning of gaze estimation

During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where antialiasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.

Adding greater realism to a computer-generated image by smoothing jagged edges within the image in an efficient manner

In an embodiment, an augmented reality display provides an expanded eye box and enlarged field of view through the use of holographic optical elements. In at least one example, an incoupling element directs an image into a waveguide, which transmits the image to a set of outcoupling gratings. In one example, a set of holographic optical elements opposite the outcoupling elements reflect the image to the user with an enlarged field of view while maintaining an expanded eye box.

Systems and methods for wide field of view augmented reality display

A remote device utilizes ray tracing to compute a light field for a scene to be rendered, where the light field includes information about light reflected off surfaces within the scene. This light field is then compressed utilizing one or more video compression techniques that implement temporal reuse, such that only differences between the light field for the scene and a light field for a previous scene are compressed. The compressed light field data is then sent to a client device that decompresses the light field data and uses such data to obtain the light field for the scene at the client device. This light field is then used by the client device to compute global illumination for the scene. The global illumination may be used to accurately render the scene at the mobile device, resulting in a realistic scene that is presented by the mobile device.

Streaming a compressed light field

The present invention facilitates efficient and effective image processing. A network can comprise: a first system configured to perform a first portion of lighting calculations for an image and combing results of the first portion of lighting calculations for the image with results of a second portion of lighting calculations; and a second system configured to perform the second portion of lighting calculations and forward the results of the second portion of the lighting calculations to the first system. The first and second portion of lighting calculations can be associated with indirect lighting calculations and direct lighting calculations respectively. The first system can be a client in a local location and the second system can be a server in a remote location (e.g., a cloud computing environment). The first system and second system can be in a cloud and a video is transmitted to a local system.

Asynchronous lighting for image illumination

The disclosure provides features or schemes that improve a user's experience with an interactive computer product by reducing latency through late latching and late warping. The late warping can be applied by imaging hardware based on late latch inputs and is applicable for both local and cloud computing environments. In one aspect, the disclosure provides a method of operating an imaging system employing late latching and late warping. In one example the method of operating an imaging system includes: (1) rendering a rendered image based on a user input from an input device and scene data from an application engine, (2) obtaining a late latch input from the input device, (3) rendering, employing imaging hardware, a warped image by late warping at least a portion of the rendered image based on the late latch input, and (4) updating state information in the application engine with late latch and warp information.

Hardware acceleration and event decisions for late latch and warp in interactive computer products

A remote device utilizes ray tracing to compute a light field for a scene to be rendered, where the light field includes information about light reflected off surfaces within the scene. This light field is then compressed utilizing lossless or lossy compression and one or more video compression techniques that implement temporal reuse, such that only differences between the light field for the scene and a light field for a previous scene are compressed. The compressed light field data is then sent to a client device that decompresses the light field data and uses such data to obtain the light field for the scene at the client device. This light field is then used by the client device to compute global illumination for the scene. The global illumination may be used to accurately render the scene at the mobile device, resulting in a realistic scene that is presented by the mobile device.

Streaming a light field compressed utilizing lossless or lossy compression

A global illumination data structure (e.g., a data structure created to store global illumination information for geometry within a scene to be rendered) is computed for the scene. Additionally, reservoir-based spatiotemporal importance resampling (RESTIR) is used to perform illumination gathering, utilizing the global illumination data structure. The illumination gathering includes identifying light values for points within the scene, where one or more points are selected within the scene based on the light values in order to perform ray tracing during the rendering of the scene.

Reservoir-based spatiotemporal importance resampling utilizing a global illumination data structure

Photon-based image illumination rendering

During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where anti-aliasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.

Adding greater realism to a computer-generated image by smoothing jagged edges

Sampling a function is used for many applications, such as rendering images. The challenge is how to select the best samples to minimize computations and produce accurate results. An alternative is to use a larger number of samples that may not be carefully selected in an attempt to increase accuracy. For a function that is an integral, such as functions used to render images, a sample distribution may be computed by inverting the integral. Unfortunately, for many integrals, it is neither easy nor practical to compute the inverted integral. Instead, warp functions may be combined to provide a sample distribution that accurately approximates the factors of the product being integrated. Each warp function approximates an inverted term of the product while accounting for the effects of warp functions approximating other factors in the product. The selected warp functions are customized or 'fitted' to implement importance sampling for the approximated product.

Estimating product integrals using a composition of warps

A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

Saccadic redirection for virtual reality locomotion

Image illumination rendering system and method

Methods are disclosed herein to blur an image to be displayed on a stereo display (such as virtual or augmented reality displays) based on the focus and convergence of the user. The methods approximate the complex effect of chromatic aberration on focus, utilizing three (R/G/B) simple Gaussian blurs. For transparency the methods utilize buffers for levels of blur rather than depth. The methods enable real-time chromatic-based blurring effects for VR/AR displays.

Computational blur for varifocal displays

Growing community of inventors

Williamstown, MA, United States of America

Morgan McGuire