Google Inc.

Cornell University

Noah Snavely

William T Freeman

Ce Liu

Forrester H Cole

Richard Tucker

Tali Dekel

Zhengqi Li

Daniel Goldman

Janne Matias Kontkanen

Rohit Kumar Pandey

Ricardo Martin Brualla

Sameh Khamis

Hugues Hoppe

Moustafa Meshry

Daniel Huttenlocher

Yungpeng Li

A method includes obtaining a reference image and a target image each representing an environment containing moving features and static features. The method also includes determining an object mask configured to mask out the moving features and preserves the static features in the target image. The method additionally includes determining, based on motion parallax between the reference image and the target image, a static depth image representing depth values of the static features in the target image. The method further includes generating, by way of a machine learning model, a dynamic depth image representing depth values of both the static features and the moving features in the target image. The model is trained to generate the dynamic depth image by determining depth values of at least the moving features based on the target image, the object mask, and the static depth image.

Depth determination for images captured with a moving camera and representing moving features

According to an aspect, a method for neural rerendering includes obtaining a three-dimensional (3D) model representing a scene of a physical space, where the 3D model is constructed from a collection of input images, rendering an image data buffer from the 3D model according to a viewpoint, where the image data buffer represents a reconstructed image from the 3D model, receiving, by a neural rerendering network, the image data buffer, receiving, by the neural rerendering network, an appearance code representing an appearance condition, and transforming, by the neural rerendering network, the image data buffer into a rerendered image with the viewpoint of the image data buffer and the appearance condition specified by the appearance code.

Neural rerendering from 3D models

A system creates an output image of a scene using two-dimensional (2D) images of the scene. For a pixel in the output image, the system identifies, in the output image, 2D fragments that correspond to the pixel. The system converts the 2D fragments into three dimensional (3D) fragments, creates volume spans for the pixel based on the 3D fragments, determines a color of a volume span based on color contribution of respective one or more of the 3D fragments for the volume span, and determines a color of the pixel for the output image from determined colors of the volume spans.

Continuous depth-ordered image compositing

System and methods for determining where a digital photograph was taken by estimating the camera pose with respect to a global scale three-dimensional database. Accurate location and orientation of the digital photograph is established through feature correspondence and geometry estimated from photograph collections.

System and methods for world-scale camera pose estimation

Growing community of inventors

New York, NY, United States of America

Noah Snavely