Google Inc.

Waymo LLC

Pratul Preeti Srinivasan

Jonathan Tilton Barron

Neal Wadhwa

Rahul Garg

Jiawen Chen

Henrik Kretzschmar

Tianfan Xue

Vincent Michael Casser

Thomas Allen Funkhouser

Andrea Tagliasacchi

Vittorio Carlo Ferrari

Andrew Huaming Liu

Konstantinos Rematas

Benjamin Joseph Mildenhall

Matthew Justin Tancik

Shumian Xin

Sabeek Mani Pradhan

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rendering a new image that depicts a scene from a perspective of a camera at a new camera viewpoint.

Large scene neural view synthesis

Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.

Radiance fields for three-dimensional reconstruction and novel view synthesis in large-scale environments

A method includes obtaining dual-pixel image data that includes a first sub-image and a second sub-image, and generating an in-focus image, a first kernel corresponding to the first sub-image, and a second kernel corresponding to the second sub-image. A loss value may be determined using a loss function that determines a difference between (i) a convolution of the first sub-image with the second kernel and (ii) a convolution of the second sub-image with the first kernel, and/or a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second kernel. Based on the loss value and the loss function, the in-focus image, the first kernel, and/or the second kernel, may be updated and displayed.

Defocus blur removal and depth estimation using dual-pixel image data

Example embodiments allow for training of artificial neural networks (ANNs) to generate depth maps based on images. The ANNs are trained based on a plurality of sets of images, where each set of images represents a single scene and the images in such a set of images differ with respect to image aperture and/or focal distance. An untrained ANN generates a depth map based on one or more images in a set of images. This depth map is used to generate, using the image(s) in the set, a predicted image that corresponds, with respect to image aperture and/or focal distance, to one of the images in the set. Differences between the predicted image and the corresponding image are used to update the ANN. ANNs tramed in this manner are especially suited for generating depth maps used to perform simulated image blur on small-aperture images.

Aperture supervision for single-view depth prediction

Growing community of inventors

San Francisco, CA, United States of America

Pratul Preeti Srinivasan