Google Inc.

Jay Yagnik

Sergey Ioffe

Shumeet Baluja

Yushi Jing

Ullas Gargi

Vivek Kwatra

Dandapani Sivakumar

Douglas Eck

Henry Allan Rowley

Ming Zhao

Alexander Toshev

Rahul Sukthankar

Mei Han

Samy Bengio

Gal Chechik

Hrishikesh Balkrishna Aradhye

Luciano Sbaiz

George Dan Toderici

Sudheendra Vijayanarasimhan

Anindya Sarkar

Matthew Nirvan Sharifi

Dragomir Dimitrov Anguelov

Abhijit S Ogale

Dominik Roblek

Gheorghe Postelnicu

David A Ross

Thomas Chadwick Walters

Dennis William Strelow

George Tzanetakis

Thomas Leung

Reto Strobl

King Hong Thomas Leung

Sangho Yoon

Juan Carlos Niebles Duque

Michele Mae Covell

Krishna Asur Bharat

Yang Song

Richard Francis Lyon

Sanjiv Kumar

Gideon S Mann

Ehud Benyamin Rivlin

Ryan Michael Rifkin

Ruei-Sung Lin

Corinna Cortes

Ameesh Makadia

Alexandru Marius Pasca

Christian Frueh

Niyati Yagnik

Ehud Rivlin

Rasmus Munk Larsen

Thomas Bugnon

Matthias Rochus Konrad

Sarvjeet Singh

Poonam Suryanarayan

Anandsudhakar Kesari

Hanna Pasula

Yi-Fan Chen

Jeff Faust

John Roberts Anderson

A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.

Video content analysis for automatic demographics recognition of users and videos

Implementations include systems and methods for scoring candidates for set recommendation problems. An example method includes repeating, for each code in code arrays for items in a set of items, determining a most common value for the code. In some implementations, the method includes determining that the most common value occurs with a frequency that meets an occurrence threshold and adding the code and the most common value to set-inclusion criteria. In other implementations, the method includes determining a value for the code from a code array for a seed item and adding the code and the most common value to set-inclusion criteria when the value for the code from the code array for the seed item matches the most common value. The method may also include evaluating a similarity with a candidate item based on the set-inclusion criteria and basing a recommendation regarding the candidate item on the similarity.

Scoring candidates for set recommendation problems

User engagement in unwatched videos is predicted by collecting and aggregating data describing user engagement with watched videos. The data are normalized to reduce the influence of factors other than the content of the videos on user engagement. Engagement metrics are calculated for segments of watched videos that indicate user engagement with each segment relative to overall user engagement with the watched videos. Features of the watched videos within time windows are characterized, and a function is learned that relates the features of the videos within the time windows to the engagement metrics for the time windows. The features of a time window of an unwatched video are characterized, and the learned function is applied to the features to predict user engagement to the time window of the unwatched video. The unwatched video can be enhanced based on the predicted user engagement.

Predicting engagement in video content

An example method is disclosed that includes identifying a training set of images, wherein each image in the training set has an identified bounding box that comprises an object class and an object location for an object in the image. The method also includes segmenting each image of the training set, wherein segments comprise sets of pixels that share visual characteristics, and wherein each segment is associated with an object class. The method further includes clustering the segments that are associated with the same object class, and generating a data structure based on the clustering, wherein entries in the data structure comprise visual characteristics for prototypical segments of objects having the object class and further comprise one or more potential bounding boxes for the objects, wherein the data structure is usable to predict bounding boxes of additional images that include an object having the object class.

System and method for using segmentation to identify object location in images

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classification using a neural network. One of the methods for processing an input through each of multiple layers of a neural network to generate an output, wherein each of the multiple layers of the neural network includes a respective multiple nodes includes for a particular layer of the multiple layers: receiving, by a classification system, an activation vector as input for the particular layer, selecting one or more nodes in the particular layer using the activation vector and a hash table that maps numeric values to nodes in the particular layer, and processing the activation vector using the selected nodes to generate an output for the particular layer.

Large-scale classification in neural networks using hashing

Provided content is determined to contain an asset represented by reference content by comparing digital fingerprints of the provided content and the reference content. The fingerprints of the reference content and the provided content are generated using a convolutional neural network (CNN). The CNN is trained using a plurality of frame triplets including an anchor frame representing the reference content, a positive frame which is a transformation of the anchor frame, and a negative frame representing content that is not the reference content. The provided content is determined to contain the asset represented by the reference content based on a similarity measure between the generated fingerprints. If the provided content is determined to contain the asset represented by the reference content, a policy associated with the asset is enforced on the provided content.

Automatic learning of a video matching system

The disclosed embodiments describe a method, an apparatus, an application specific integrated circuit, and a server that provides a fast and efficient look up for data analysis. The apparatus and server may be configured to obtain data segments from a plurality of input devices. The data segments may be individual unique subsets of the entire data set obtained by a plurality input devices. A hash function may be applied to an aggregated set of the data segments. A result of the hash function may be stored in a data structure. A codebook may be generated from the hash function results.

Fast efficient vocabulary computation with hashed vocabularies

This disclosure relates to audio identification using ordinal transformations. A media matching component receives a sample audio file. The sample audio file can include, for example, a cover song. The media matching component includes a vector component that computes a set of vectors using auditory feature values included in the sample audio file. A hashing component employs a hash function to generate a fingerprint, including a set of sub-fingerprints, for the sample audio file using the set of vectors. The fingerprint is invariant to variations including but not limited to variations in key, instrumentation, encoding formats, performers, performance conditions, arrangement, and/or recording and processing variations. An identification component determines if any reference audio files are similar to the sample audio file using the fingerprint and/or sub-fingerprints, and identifies any similar reference audio files.

Audio identification using ordinal transformation

A segmentation annotation technique for media items is disclosed herein. Given a weakly labeled media item, spatiotemporal masks may be generated for each of the concepts with which it is labeled. Segments may be ranked by the likelihood that they correspond to a given concept. The ranked concept segments may be utilized to train a classifier that, in turn, may be used to classify untagged or new media items.

Classification using concept ranking according to negative exemplars

One of the described methods includes receiving a plurality of images from a camera, the plurality of images comprising a sequence; identifying one or more two-dimensional features in each of a plurality of images in the received sequence of images; associating a three-dimensional point with each of the identified one or more two-dimensional features; tracking each of the one or more two-dimensional features through successive images in the plurality of images; and iteratively minimizing a two-dimensional image error between the tracked each of the one or more two-dimensional features and an image reprojection with respect to the three-dimensional point corresponding to the one or more two-dimensional features and a three-dimensional position of the camera corresponding to one or more of the plurality of images.

General and nested Wiberg minimization

A method includes identifying a named entity, retrieving images associated with the named entity, and using a face detection algorithm to perform face detection on the retrieved images to detect faces in the retrieved images. At least one representative face image from the retrieved images is identified, and the representative face image is used to identify one or more additional images representing the at least one named entity.

Identifying images using face recognition

A system and methods for automatically detecting temporal music trends by observing music consumption by users of online services, for example, social networks, and user sharing habits. In some embodiments, the system and methods gather music consumption patterns (e.g., downloading, listening, sharing or the like) of users, including music identifiers for a track, album, or playlist in a user's music library and time stamps that indicate consumption times corresponding to the music identifiers. A temporal trends detection engine determines music of interest to users by analyzing music consumption patterns of users, user interests and tastes in music, and social affinity between users. A recommendations engine automatically generates and transmits recommendations of music determined by the temporal trends detection engine to be of interest to users.

System and methods for detecting temporal music trends from online services

This disclosure relates to transformation invariant media matching. A fingerprinting component can generate a transformation invariant identifier for media content by adaptively encoding the relative ordering of interest points in media content. The interest points can be grouped into subsets, and stretch invariant descriptors can be generated for the subsets based on ratios of coordinates of interest points included in the subsets. The stretch invariant descriptors can be aggregated into a transformation invariant identifier. An identification component compares the identifier against a set of identifiers for known media content, and the media content can be matched or identified as a function of the comparison.

Transformation invariant media matching

Growing community of inventors

Mountain View, CA, United States of America

Jay Yagnik