Google Inc.

Daniel William Erickson

Paul E Debevec

Manfred Ernst

Harrison McKenzie Chapter

Sofien Bouaziz

Shahram Izadi

John Patrick Flynn

Johnny Chung Lee

Evan Hardesty Parker

Jonathan James Taylor

Ryan Styles Overbeck

Dominik Philemon Kaeser

Sean Ryan Francesco Fanello

Philip L Davidson

Rohit Kumar Pandey

Christoph Rhemann

Matthew Milton Pharr

Yinda Zhang

Chloe LeGendre

Christian Haene

Jessica Lynn Busch

Jason Angelo Dourgarian

Sergio Orts Escolano

Craig Donner

Matthew Whalen

Thabo Beeler

Michael Joseph Broxton

Lars Peter Johannes Hedman

Matthew Nowicki DuVall

Peter Barnum

Abhi Meka

Ryan Overbeck

Harrison Mckenzie Chapter

Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

Volumetric performance capture with neural rendering

Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.

Methods, systems, and media for generating an immersive light field video with a layered mesh representation

An HMD device identifies a pose of the device and identifies a subset of a plurality of camera viewpoints of a light-field based on the pose. The HMD device interpolates image data of the light-field based on the pose and the subset of the plurality of camera viewpoints to generate an interpolated view; and displays at the HMD device an image based on the interpolated view. By interpolating based on the subset of camera viewpoints, the HMD device can reduce processing overhead and improve the user experience.

Light-field viewpoint and pixel culling for a head mounted display device

In a system having a user-portable display device, a method includes maintaining a lightfield data structure representing at least a portion of a four-dimensional (4D) lightfield for a three-dimensional (3D) world in association with a first pose of the user-portable display device relative to the 3D world. The method further includes determining a second pose of the user-portable display device relative to the 3D world, the second pose comprising an updated pose of the user-portable display device. The method additionally includes generating a display frame from the lightfield data structure based on the second pose, the display frame representing a field of view of the 3D world from the second pose.

System and methods for performing electronic display stabilization via retained lightfield rendering

Systems and method relating to creating a virtual reality, such as a three-dimensional virtual reality, representation of physical scene. In this aspect, such a method may comprise gathering information from an array of cameras positioned on a two-dimensional planar surface. In this particular aspect, one or more of the cameras may be positioned at a different angle relative to the two-dimensional planar surface based at least in part on a respective distance of each of the one or more cameras from a midpoint of the planar surface. Furthermore, in this general aspect the method may further comprise processing the gathered information at least in part to render a virtual reality representation of the physical scene.

Capture and render of virtual reality content employing a light field camera array

Growing community of inventors

San Francisco, CA, United States of America

Daniel William Erickson