Nvidia Corporation

International Business Machines Corporation

David Patrick Luebke

Douglas Robert Lanman

Samuli Matias Laine

Morgan McGuire

Timo Oskari Aila

Gerrit Ary Slavenburg

Michael Stengel

Jonghyun Kim

Tero Tapani Karras

Thomas Francis Fox

Michael Garland

Ward Lopes

Jaakko Tapani Lehtinen

Peter Schuyler Shirley

Miika Samuli Aittala

Ting-Chun Wang

Arun Mohanray Mallya

Joohwan Kim

Kaan Aksit

Michael Mara

Jan Kautz

Ming-Yu Liu

Aaron Eliot Lefohn

Christopher Ryan Wyman

Anjul Patney

Qi Sun

Peter-Pike Sloan

Brent Leonard Oster

John R Nickolls

Marco Salvi

Brian Matthew Fahs

Peter C Mills

Lars Nyland

Jeremy Sugerman

John A Stratton

Cyril Crassin

Timothy Foley

Omer Shapira

Christian Lauterbach

Douglas A Voorhies

Kari Pulli

Shalini De Mello

Jacopo Pantaleoni

Jonathan Cohen

Morgan Samuel McGuire

Dawid Stanislaw Pajak

Eric Michael Whitmire

Felix Heide

Craig E Kolb

Fu-Chung Huang

Dikpal Reddy

Ben Boudaoud

Cyril Jean-Francois Crassin

Alessandro Dal Corso

Youngmo Jeong

Manu Gopakumar

Eugene J d'Eon

Qi Mo

Jonathan Bakdash

Zander Majercik

Joseph S Stam

David Cook

Jai P Menon

Wolfgang Heidrich

David Kirk McAllister

Anton S Kaplanyan

Thomas Hastings Greer

Alexander V Reshetov

Josef Bo Spjut

James Thomas Klosowski

Eric Enderton

David Tarjan

Scott Saulters

Alexander Majercik

Andrew Stephen Maimone

Jui-Yi Wu

Morgan Mcguire

Jihad El-Sana

Fausto Berdardini

Jared Hoberock

Michael Robert Phillips

Nir Benty

Chengyuan Lin

Pete Shirley

Apparatuses, systems, and techniques to perform compression of video data using neural networks to facilitate video streaming, such as video conferencing. In at least one embodiment, a sender transmits to a receiver a key frame from video data and one or more keypoints identified by a neural network from said video data, and a receiver reconstructs video data using said key frame and one or more received keypoints.

Video compression using neural networks

One embodiment of a method includes calculating one or more activation values of one or more neural networks trained to infer eye gaze information based, at least in part, on eye position of one or more images of one or more faces indicated by an infrared light reflection from the one or more images.

Synthetic infrared image generation for machine learning of gaze estimation

One embodiment of a display system includes one or more light sources, one or more spatial light modulators, and a plurality of scatterers. One embodiment of a method for displaying content includes computing at least one of a phase or an amplitude modulation associated with two-dimensional (2D) or three-dimensional (3D) content, and causing one or more spatial light modulators to modulate light based on the at least one of a phase or an amplitude modulation to generate modulated light, where the modulated light is scattered by a plurality of scatterers.

Holographic volumetric displays

Apparatuses, systems, and techniques are presented to estimate user gaze. In at least one embodiment, one or more neural networks are used to determine coarse and fine gaze estimates for one or more users.

Gaze estimation using one or more neural networks

The disclosure provides a method for audio calibration that uses audio simulation and reconstructed surface information from images or video recordings along with recorded sound. The surface component of the method introduces knowledge that enables audio wave propagation simulation for a particular location. Using the simulation results the sound distribution can be optimized. For example, unwanted audio reflection and occlusion can be recognized and resolved. In one example, the disclosure provides a method for improving acoustics at a location that includes: (1) generating a geometric model of a location using visual data obtained from the location, wherein the location includes an audio system, and (2) simulating, using the geometric model, movement of sound waves in the location that originate from the audio system. The disclosure also provides a computer system, a computer program product, and a mobile computing device that include features for improving acoustics at a location.

Vision-based sound simulation for correcting acoustics at a location

Virtual reality (VR) displays are computer displays that present images or video in a manner that simulates a real experience for the viewer. In many cases, VR displays are implemented as head-mounted displays (HMDs) which provide a display in the line of sight of the user. Because current HMDs are composed of a display panel and magnifying lens with a gap therebetween, proper functioning of the HMDs limits their design to a box-like form factor, thereby negatively impacting both comfort and aesthetics. The present disclosure provides a different configuration for a virtual reality display which allows for improved comfort and aesthetics, including specifically at least one coherent light source, at least one holographic waveguide coupled to the at least one coherent light source to receive light therefrom, and at least one spatial light modulator coupled to the at least one holographic waveguide to modulate the light.

Holographic virtual reality display

The present invention facilitates efficient and effective image processing. A network can comprise: a first system configured to perform a first portion of lighting calculations for an image and combing results of the first portion of lighting calculations for the image with results of a second portion of lighting calculations; and a second system configured to perform the second portion of lighting calculations and forward the results of the second portion of the lighting calculations to the first system. The first and second portion of lighting calculations can be associated with indirect lighting calculations and direct lighting calculations respectively. The first system can be a client in a local location and the second system can be a server in a remote location (e.g., a cloud computing environment). The first system and second system can be in a cloud and a video is transmitted to a local system.

Asynchronous lighting for image illumination

Virtual reality (VR) displays are computer displays that present images or video in a manner that simulates a real experience for the viewer. In many cases, VR displays are implemented as head-mounted displays (HMDs) which provide a display in the line of sight of the user. Because current HMDs are composed of a display panel and magnifying lens with a gap therebetween, proper functioning of the HMDs limits their design to a box-like form factor, thereby negatively impacting both comfort and aesthetics. The present disclosure provides a different configuration for a VR display which allows for improved comfort and aesthetics, including specifically at least one coherent light source, at least one pupil replicating waveguide coupled to the at least one coherent light source to receive light therefrom, and at least one spatial light modulator coupled to the at least one pupil replicating waveguide to modulate the light.

In an embodiment, an augmented reality display is provided that incorporates a prescription lens for the wearer. In an embodiment, an image is generated from a display and directed into the edge of the prescription lens, and the lens acts as a waveguide. The image is internally reflected within the prescription lens, and is directed to the wearer by an image combiner embedded within the prescription lens. In an embodiment, the augmented reality display can be adjusted for many common vision problems including myopia, hyperopia, astigmatism, and presbyopia.

Prescription augmented reality display

A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.

Generative adversarial neural network assisted video reconstruction

An augmented reality display system includes a first beam path for a foveal inset image on a holographic optical element, a second beam path for a peripheral display image on the holographic optical element, and pupil position tracking logic that generates control signals to set a position of the foveal inset as perceived through the holographic optical element, to determine the peripheral display image, and to control a moveable stage.

Foveated display for augmented reality

Generative adversarial neural network assisted compression and broadcast

Generative adversarial neural network assisted reconstruction

Generative adversarial neural network assisted video compression and broadcast

Growing community of inventors

San Jose, CA, United States of America

David Patrick Luebke