Google Inc.

Richard Tucker

William T Freeman

Ce Liu

Forrester H Cole

Zhengqi Li

Gheorghe Postelnicu

Tali Dekel

Noah Snavely

Michele Mae Covell

David Marwood

Sergey Ioffe

Behshad Behzadi

Monika Hildegard Henzinger

Daniel M Keysers

Varun Kacholia

Yaniv Bernstein

Jasmine Kent

Dominik Philemon Kaeser

Nicola Muscettola

Keith Noah Snavely

Benjamin Liebald

Michael Krainin

Huiwen Chang

Murali Krishna Viswanathan

David Salesin

Brian Curless

Abhishek Kar

Kyle Sargent

Henrik Rydgard

Stefan Burkhardt

Varun Jampani

Qianqian Wang

Brain L Curless

Dominik Kaeser

William T Frreeman

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rendering a new image that depicts a scene from a perspective of a camera at a new camera viewpoint at a given time point in a video.

Neural dynamic image-based rendering

A method includes determining, based on an image having an initial viewpoint, a depth image, and determining a foreground visibility map including visibility values that are inversely proportional to a depth gradient of the depth image. The method also includes determining, based on the depth image, a background disocclusion mask indicating a likelihood that pixel of the image will be disoccluded by a viewpoint adjustment. The method additionally includes generating, based on the image, the depth image, and the background disocclusion mask, an inpainted image and an inpainted depth image. The method further includes generating, based on the depth image and the inpainted depth image, respectively, a first three-dimensional (3D) representation of the image and a second 3D representation of the inpainted image, and generating a modified image having an adjusted viewpoint by combining the first and second 3D representation based on the foreground visibility map.

Single image 3D photography with soft-layering and depth-aware inpainting

A method includes obtaining a reference image and a target image each representing an environment containing moving features and static features. The method also includes determining an object mask configured to mask out the moving features and preserves the static features in the target image. The method additionally includes determining, based on motion parallax between the reference image and the target image, a static depth image representing depth values of the static features in the target image. The method further includes generating, by way of a machine learning model, a dynamic depth image representing depth values of both the static features and the moving features in the target image. The model is trained to generate the dynamic depth image by determining depth values of at least the moving features based on the target image, the object mask, and the static depth image.

Depth determination for images captured with a moving camera and representing moving features

Systems and methods are provided herein relating to real-time duplicate detection of video content. Fingerprints can be generated for an uploaded video. The fingerprints can be used to match the uploaded video to a set of matching videos. The set of matching videos can be filtered based on the type of match, and the quality of the match. A unique cluster-id can be generated for the uploaded video containing an upload time, and that unique cluster-id can then be modified to associate the uploaded video with a cluster-id of potential duplicates. Cluster-ids can then be used in the context of a search to filter results that have identical cluster-ids. The benefits in using real-time duplicate detection can better maximize user experiences in a video sharing service that contains potential duplicates of the same content.

Real-time duplicate detection for uploaded videos

A combined fingerprint is generated for a video that can match two near-identical videos that differ only in their aspect ratios or formats. A transformation strategy is selected by selecting a first and a second aspect correction method. A first transformed video is generated by applying the first aspect correction method to the video. A second transformed video is generated by applying the second aspect correction method to the video. A first fingerprint is generated using the first transformed video. A second fingerprint is generated using the second transformed video. The combined fingerprint is generated by combining the first half of the first fingerprint with the second half of the second fingerprint.

Fingerprinting to match videos having varying aspect ratios

A system and method of use are provided for generating video fingerprints that are robust to rotation, stretching, scaling, flipping and other affine transformations. The system and method are based on determining points of interest for a target video image and a collection of images stored in a database. A method of identifying a copied video image is also provided through comparison of the fingerprint of the target video image with fingerprints of images in a database. The method of video fingerprinting involves: defining points of interest from a video image, determining an oriented bounding area encompassing said points of interest that have been transformed by the same transformations used to produce the oriented bounding area and creating a fingerprint by encoding the points of interest as Gray Code such that similar images have a small Hamming distance and dissimilar images have a large Hamming distance. The identification of points of interest are provided by a number of techniques including segmentation, identification of affine covariant regions and scale invariant features.

Fingerprinting image using points of interest for robust image identification

Large-scale matching of videos is performed by matching a set of probe videos against a set of reference videos to determine if they are visually and/or aurally similar. The visual and audio fingerprints of all probe videos and reference videos are divided into subfingerprints, which are divided into LSH bands. The LSH bands of the probe videos are sorted in one list, and the LSH bands of the reference videos are sorted in another list. Then, the two sorted lists are linearly scanned for matching LSH bands. The matching LSH bands are sorted by probe video ID, and each probe video ID is searched to find matches between probe videos and reference videos. Further, an incremental matching process identifies matches as groups of new probe videos and/or new reference videos are added, without unnecessary repetition of matching old probe videos against old reference videos.

Large-scale matching of audio and video

The present disclosure includes, among other things, systems, methods and program products for selecting subsequences (shingles or tuples) generated from sequences of tokens.

Algorithms for selecting subsequences

Growing community of inventors

New York, NY, United States of America

Richard Tucker