Google Inc.

Zhen Li

Futang Peng

Aleksei Timofeev

Chun-Ta Lu

Andrew Stephen Tomkins

Hrishikesh Balkrishna Aradhye

Sujith Ravi

Thomas J Duerig

Huazhong Ning

Da-Cheng Juan

Yi-ting Chen

Yaxi Gao

Ariel Fuxman

Chen Sun

Teresa Ko

Jason Chang

Lily Berg

Kristina Bohl

David William Hendon

Ivan Oropeza

Ethan Schreiber

Tracy Gu

Howard Zhou

Shanfeng Zhang

Manan Shah

Chao Jia

Krishnamurthy Viswanathan

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.

Training image and text embedding models

A media application determines, based on pixels of images or videos from a collection of media items, clusters of media items such that the media items in each cluster have a visual similarity, wherein the collection of media items is associated with a user account. The media application selects a subset of the clusters of media from corresponding clusters of media items based on the media items in each cluster having a visual similarity within a range of threshold similarity values. The media application causes a user interface to be displayed that includes the subset of the clusters of media.

Determining a visual theme in a collection of media items

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for realizing a multimodal image classifier. In an aspect, a method includes, for each image of a plurality of images: processing the image by a textual generator model to obtain a set of phrases that are descriptive of the content of the image, wherein each phrase is one or more terms, processing the set of phrases by a textual embedding model to obtain an embedding of predicted text for the image, and processing the image using an image embedding model to obtain an embedding of image pixels of the image. Then a multimodal image classifier is trained on the embeddings of predicted text for the images and the embeddings of image pixels for the images to produce, as output, labels of an output taxonomy to classify an image based on the image as input.

Multimodal image classifier using textual and visual embeddings

A method includes generating a ranking model and a baseline mixing weight for each latent user category from a plurality of latent user categories based on a community preference dataset and one or more latent variables that relate the users from the community of users to the latent user categories. The method also includes generating a personalized mixing weight for each latent user category for a specified user based on an individual preference dataset, the ranking models for the latent user category, and one or more latent variables that relate the specified user to the latent user categories. The method also includes adjusting the personalized mixing weight for each latent user category for the specified user based on the baseline mixing weights, and generating ranking output for at least some objects from the plurality of objects using the personalized mixing weights and the ranking models.

Latent user models for personalized ranking

A system, apparatus, and method for video categorization using heterogeneous signals are disclosed. One aspect of the disclosed embodiments is a method for categorizing a plurality of video streams. The method includes determining a plurality of features of at least some of the plurality of video streams, determining a category of a first video stream of the plurality of video stream based on its plurality of features, identifying, using a processor, a relationship between the first video stream and a second video stream of the plurality of videos, the relationship having an associated weight, and updating, using the processor, the category of the first video stream based on a first message determined using the associated weight.

Video categorization using heterogeneous signals

Growing community of inventors

Urbana, IL, United States of America

Zhen Li