Google Inc.

Marius Renn

Rodrigo Lima Carceroni

Wei Hong

Wei Hua

Radford Ray Juang

Andrew Gerritt Huibers

Ruei-Sung Lin

Pannag Raghunath Sanketi

Alan Newberger

Jacob Mintz

Sascha Häberling

Hrishikesh Balkrishna Aradhye

Huazhong Ning

Chun-Te Chu

Li Wei

Xuhui Jia

Yukun Zhu

Shengyi Lin

Bradley Ray Green

Nisha Masharani

Sushil Nath

Emre Sargin

Leslie Tsu-Hwa Shaw

Yandong Li

Yu-Chuan Su

Yang Zhao

An example method includes outputting, by a computing device for display, one or more digital images that include a face of a user, receiving, by the computing device and based on a facial recognition process, an indication of a match between facial data associated with the face of the user and reference facial data associated with a face of an enrolled user of the computing device, and retrieving, by the computing device, digital makeup enhancement data that is associated with the reference facial data. The example method further includes applying, by the computing device, the digital makeup enhancement data to the facial data, of tire face of the user to generate one or more modified digital images that indicate at least one corresponding digital makeup enhancement to the face of the user, and outputting, by the computing device for display, the one or more modified digital images.

Applying stored digital makeup enhancements to recognized faces in digital images

Systems and methods for training a restoration model can leverage training for two sub-tasks to train the restoration model to generate realistic and identity-preserved outputs. The systems and methods can balance the training of the generation task and the reconstruction task to ensure the generated outputs preserve the identity of the original subject while generating realistic outputs. The systems and methods can further leverage a feature quantization model and skip connections to improve the model output and overall training.

Training a restoration model for balanced generation and reconstruction

The present disclosure provides systems and methods that use and/or generate image files according to a novel microvideo image format. For example, a microvideo can be a file that contains both a still image and a brief video. The microvideo can include multiple tracks, such as, for example, a separate video track, audio track, and/or one or more metadata tracks. As one example track, the microvideo can include a motion data track that stores motion data that can be used (e.g., at file runtime) to stabilize the video frames. A microvideo generation system included in an image capture device can determine a trimming of the video on-the-fly as the image capture device captures the microvideo.

Microvideo system, format, and method of generation

In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for detecting motion in images. A computing system receives first and second images that were captured by a camera. The computing system generates, using the images, a mathematical transformation that indicates movement of the camera from the first image to the second image. The computing system generates, using the first image and the mathematical transformation, a modified version of the first image that presents the scene that was captured by the first image from a position of the camera when the second image was captured. The computing system determines a portion of the first image or second image at which a position of an object in the scene moved, by comparing the modified version of the first image to the second image.

Detecting motion in images

Techniques described herein include receiving a first video frame including a first color map, receiving at least one second video frame, receiving a third video frame including a second color map, generating at least one color map based on the first color map and the second color map, associating the at least one third color map with the at least one second frame, the associated at least one second frame being sequentially positioned between the first frame and the second frame, tone correcting the associated at least one second frame based on the at least one generated third color map and color correcting the at least one second frame based on the at least one generated third color map.

Video tone mapping using a sequence of still images

Implementations of the disclosed technology include techniques for autonomously collecting image data, and generating photo summaries based thereon. In some implementations, a plurality of images may be autonomously sampled from an available stream of image data. For example, a camera application of a smartphone or other mobile computing device may present a live preview based on a stream of data from an image capture device. The live stream of image capture data may be sampled and the most interesting photos preserved for further filtering and presentation. The preserved photos may be further winnowed as a photo session continues and an image object generated summarizing the remaining photos. Accordingly, image capture data may be autonomously collected, filtered, and formatted to enable a photographer to see what moments they missed manually capturing during a photo session.

Systems and methods for autonomously generating photo summaries

The disclosed technology includes techniques for improved content coverage in automatically-generated content summaries. The technique may include clustering a set of input content, determining diffusion for each cluster, and selecting representatives of each cluster to optimize other secondary metrics. Various types of input content may be used, including groups of images, video clips, or other multimedia content. Contiguous content may be manually or programmatically divided into discrete portions before clustering, for example, a lengthy video divided into a number of short clips. In some implementations, the disclosed technique may be implemented effectively on a mobile device. In other words, the processing required may be computationally feasible for execution on a smartphone or similar device.

Systems and methods for improved coverage of input media in content summarization

Techniques for determining motion saliency in video content using center-surround receptive fields. In some implementations, images or frames from a video may be apportioned into non-overlapped regions, for example, by applying a rectilinear grid. For each grid region, or cell, motion consistency may be measured between the center and surround area of that cell across frames of the video. Consistent motion across the center-surround area may indicate that the corresponding region has low variation. The larger the difference between center-surround motions in a cell, the more likely the region has high motion saliency.

Systems and methods for determining motion saliency

Disclosed are methods for automatically generating commands to transform a video sequence based on information regarding speaking participants derived from the audio and video signals. The audio stream is analyzed to detect individual speakers and the video is optionally analyzed to detect lip movement to determine a probability that a detected participant is speaking. Commands are then generated to transform the video stream consistent with the identified speaker.

Method and apparatus for focus-of-attention control

Growing community of inventors

San Jose, CA, United States of America

Marius Renn