Google Inc.

Jingbin Wang

Charles Joseph Rosenberg

Xiangrong Chen

Sarah Moussa

Anelia Angelova

Yi Shen

Michele Mae Covell

Yushi Jing

Hui Li

Erik Murphy-Chutorian

Andrea Lynn Frome

Radhika Malpani

Weilong Yang

Yoram Singer

Zheng Sun

Yun Shi

Sean O'Malley

Tianpeng Jin

Ce Liu

Peyman Milanfar

Feng Yang

Dilip Krishnan

Mikael Bonnevie

Min-Hsuan Tsai

Carolyn F Au

Michael Krainin

Henrik C Stewenius

Aaron Joseph Maschinot

Hossein Talebi

Min-hsuan Tsai

Piotr Teterwak

Aaron Yehuda Sarna

James Futrell

Sean O Malley

Shuchao Bi

Wenchao Tong

Haifeng Gong

Raanan Sayag

Michael Spencer Krainin

Mikael Pierre Bonnevie

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating realistic extensions of images. In one aspect, a method comprises providing an input that comprises a provided image to a generative neural network having a plurality of generative neural network parameters. The generative neural network processes the input in accordance with trained values of the plurality of generative neural network parameters to generate an extended image. The extended image has (i) more rows, more columns, or both than the provided image, and (ii) is predicted to be a realistic extension of the provided image. The generative neural network is trained using an adversarial loss objective function.

Image extension neural networks

Systems and methods of automatically extracting summaries of video content are described herein. A data processing system can access, from a video database, a first video content element including a first plurality of frame. The data processing system can select an intervallic subset of the first plurality of frames of the first video content element. The data processing system can calculate, for each of a plurality of further subsets comprising a predetermined number of frames from the intervallic subset, a score for the further subset. The data processing system can identify, from the plurality of further subsets, a further subset having a highest score. The data processing system can select a portion of the first video content element comprising the frames of the further subset having the highest score. The data processing system can generate a second video content element comprising the selected portion of the first video content element.

Summarizing video content

Methods and apparatus to reduce compression artifacts in images are disclosed herein. An example method includes separating at least a portion of an image into a first component and a second component, reducing a first artifact in the first component to form a first cleaned component, reducing, using the first cleaned component, a second artifact in the second component to form a second cleaned component, and combining the first cleaned component and the second cleaned component to form a cleaned image.

Methods and apparatus to reduce compression artifacts in images

Methods, systems, and apparatus, include computer programs encoded on a computer-readable storage medium, for determining keywords for an image that supports an overlay content item. A method includes identifying, using one or more processors, an image that is to support an overlay content item, the image being presented on a web site and including a portion that is designated as being enabled to receive and display the overlay content item; evaluating pixel data associated with the image including determining one or more labels that are associated with content included within the image; and determining one or more keywords for the image based at least in part on the one or more labels.

Content selection based on image content

Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Choosing image labels

Methods, systems and apparatus for identifying result images for a query image. One or more labels that are associated with the query image are obtained. Candidate images matching the query labels are identified. Visual similarity scores are generated for the candidate images. Each visual similarity score represents the visual similarity of a respective candidate image to the query image. Relevance scores are generated for each of the candidate images based on the visual similarity scores. Each relevance score represents a measure of relevance of the respective candidate images to the query image. The candidate images are ranked based on the relevance scores, a highest ranking subset of the candidate images being identified as result images and referenced by image search results. The result images can be candidate images that satisfy a similarity condition relative to the query image and other result images.

Query image search

In general, in one aspect, a method includes receiving, by the one or more processors, from a client device, data specifying a query image, requesting, by the one or more processors, text labels responsive to the query image from a database of text labels associated with images, in response to no zero text labels that meet a threshold being returned for the request, sending, by the one or more processors, request data to the client device that causes the client device to display a prompt to enter a textual search query, receiving, in response to the request data, a textual search query from the client device, and in the database of text labels associated with images, associating, by the one or more processors, the textual search query with the query image. This and other embodiments of this aspect can include corresponding systems, apparatus, and computer program products.

In general, in one aspect, a method includes receiving, by one or more processors, a query image, obtaining, by the one or more processors, search results responsive to the query image, each search result identifying a corresponding web page containing an image that is determined to be responsive to the query image, the search results being ordered in a first arrangement, obtaining, by the one or more processors, text labels associated with the query image, and generating, by the one or more processors a second arrangement of the search results by ordering the search results based on a presence in the respective corresponding web pages of one or more of the text labels associated with the query image. This and other implementations of this aspect can include corresponding systems, apparatus, and computer program products.

In general, in one aspect, a method includes receiving a query image, obtaining a set of text labels associated with the query image, each text label associated with a confidence score that the text label accurately describes a topic of the query image, selecting a first text label associated with the query image, the first text label having a highest confidence score among the set of text labels, selecting a subset of the set of text labels that have a confidence score greater than a confidence score threshold, obtaining search results responsive to the query image, and in response to at least one of the search results identifying a web page containing the first text label, sending data that causes a client device to display the subset of text labels with the search results.

Methods, systems, and apparatus, including computer program products, for ranking images are disclosed. An image search subsystem generates an adjustment factor representative of a quality measure of an image relative to a search query. The quality represents a relevance of the image to the query. The adjustment factor can be computed based on relevance data for the image to the query and image similarity data representing a relative similarity between the image and other images relevant to the query. The relevance data can be based on user actions in response to the image being included in search results for the query. The adjustment factor can be scaled based on whether the relevance data and the image similarity data both indicate that the image is relevant to the search query. A relevance score is computed based on the adjustment factor (e.g., a product of the adjustment factor and relevance score).

Ordering image search results

Method, system, and computer readable medium for identifying result images based on an image query

Growing community of inventors

Mountain View, CA, United States of America

Jingbin Wang