Google Inc.

Matthew Whalen

Paul E Debevec

Jessica Lynn Busch

Jason Angelo Dourgarian

Sofien Bouaziz

Shahram Izadi

John Patrick Flynn

Adarsh Prakash Murthy Kowdle

Jonathan James Taylor

Ryan Styles Overbeck

Sean Ryan Francesco Fanello

Christoph Rhemann

Rohit Kumar Pandey

Graham Fyffe

Julien Pascal Christophe Valentin

Andrea Tagliasacchi

Daniel William Erickson

Xueming Yu

Geoffrey Douglas Harvey

Peter Christopher Lincoln

Peter Joseph Denny

Michael Joseph Broxton

Lars Peter Johannes Hedman

Matthew Nowicki DuVall

Abhimitra Meka

Christian Häne

Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.

Methods, systems, and media for generating an immersive light field video with a layered mesh representation

Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.

Methods, systems, and media for relighting images using predicted deep reflectance fields

Growing community of inventors

San Clemente, CA, United States of America

Matthew Whalen