Google Inc.

Roblox Corporation

Loomai, Inc.

Vivek Verma

Kiran Bhat

Will Welch

Alan Sheridan

Scott Benjamin Satkin

Inaki Navarro

Dario Kneubuhler

Eloi Du Bois

Ian Sachs

Tijmen Verhulsdonck

Onur Gonen Guleryuz

Jonathan Tilton Barron

Shahram Izadi

Neal Wadhwa

Adarsh Prakash Murthy Kowdle

Sean Ryan Francesco Fanello

Vladimir Tankovich

Christoph Rhemann

Ambrus Csaszar

Stephen Charles Hsu

Konstantine Nicholas John Tsotsos

Michael John Schoenberg

Sameh Khamis

Mirko Schmidt

Eric Turner

Arthur Robert Pope

Julien Valentin

Ivan Dryanovski

Maksym Dzitsiuk

Mira Leung

Jose Pascoal

João Manuel Castro Afonso

Yanxi Liu

Chong Shang

Eloi Henri Homere Du Bois

Rishabh Battulwar

Implementations described herein relate to methods, systems, and computer-readable media to generate animations for a 3D avatar from input video captured at a client device. A camera may capture video of a face while a trained face detection model and a trained regression model output a set of FACS weights, head poses, and facial landmarks to be translated into the animations of the 3D avatar. Additionally, a higher level-of-detail may be intelligently selected based upon user preferences and/or computing conditions at the client device.

Robust facial animation from video using neural networks

Aspects of the disclosure generally relate to connecting panoramic images. One or more computing devices may load and display a first panoramic image captured at a first location and receive a selection of an area on the first panoramic image, the area corresponding to where a connection to other panoramic images may be made. The one or more computing devices may identify and display one or more nearby panoramic images which were captured near the first location and receive a selection of one of the one or more nearby panoramic images. The one or more computing devices may display the selected panoramic image and the first panoramic image and align the selected panoramic image with the first panoramic image such that the selected panoramic image is oriented in the same direction as the first panoramic image. The one or more computing devices may connect the selected panoramic image with the first panoramic image.

Method and system for obtaining pair-wise epipolar constraints and solving for panorama pose on a mobile device

Systems and methods for animating from audio in accordance with embodiments of the invention are illustrated. One embodiment includes a method for generating animation from audio. The method includes steps for receiving input audio data, generating an embedding for the input audio data, and generating several predictions for several tasks from the generated embedding. The several predictions includes at least one of blendshape weights, event detection, and/or voice activity detection. The method includes steps for generating a final prediction from the several predictions, where the final prediction includes a set of blendshape weights, and generating an output based on the generated final prediction.

Systems and methods for animation generation

A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.

Depth from motion for augmented reality for handheld user devices

The aspects described herein relate to replacing pixels in images in order to remove obstructions from images. In one example, an image of a scene having symmetrical features and image information identifying at least one pixel of the image corresponding to a hole to be filled may be received. This hole may correspond to an obstruction in the image. A set of symmetry axes may be identified based on the symmetrical features. A symmetry map identifying correspondences between different pixels in the image based on the set of symmetry axes may be generated. A correspondence between the at least one pixel corresponding to a hole to be filled and a second pixel of the image is identified based at least in part on the symmetry map and the image information. The at least one pixel may be altered based on the identified correspondence in order to remove the obstruction.

Symmetry-based interpolation in images

Growing community of inventors

Oakland, CA, United States of America

Vivek Verma