Google Inc.

Other

Rambus Inc.

Ecole Polytechnique Federale De Lausanne

Luciano Sbaiz

Jesse Berent

Jay Yagnik

Hrishikesh Balkrishna Aradhye

Martin Vetterli

George Dan Toderici

Reto Strobl

Effrosyni Kokiopoulou

Dimitre Trendafilov

Juan Carlos Niebles Duque

Thomas Chadwick Walters

Alexandru Marius Pasca

Edoardo Charbon

Serge Ayer

King Hong Thomas Leung

Thomas Bugnon

Thomas Fischbacher

Sabine Susstrunk

Matthias Rochus Konrad

Andrea Gesmundo

Hanna Pasula

Emmanuel Reusens

Jasper Snoek

Daniel Filip

Paolo Prandoni

Krzysztof Stanislaw Maziarz

Gábor Bartók

Systems and methods for flexible parameter sharing for multi-task learning are provided. A training method can include obtaining a test input, selecting a particular task from one or more tasks, and training a multi-task machine-learned model for the particular task by performing a forward pass using the test input and one or more connection probability matrices to generate a sample distribution of test outputs, training the components of the machine-learned model based at least in part on the sample distribution, and performing a backwards pass to train a connection probability matrix of the multi-task machine-learned model using a straight-through Gumbel-softmax approximation.

Flexible parameter sharing for multi-task learning

A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.

Video content analysis for automatic demographics recognition of users and videos

Provided content is determined to contain an asset represented by reference content by comparing digital fingerprints of the provided content and the reference content. The fingerprints of the reference content and the provided content are generated using a convolutional neural network (CNN). The CNN is trained using a plurality of frame triplets including an anchor frame representing the reference content, a positive frame which is a transformation of the anchor frame, and a negative frame representing content that is not the reference content. The provided content is determined to contain the asset represented by the reference content based on a similarity measure between the generated fingerprints. If the provided content is determined to contain the asset represented by the reference content, a policy associated with the asset is enforced on the provided content.

Automatic learning of a video matching system

A system for automated annotation of images and videos points a mobile device towards an object of interest, such as a building or landscape scenery, for the device to display an image of the scene with an annotation for the object. An annotation can include names, historical information, and links to databases of images, videos, and audio files. Different techniques can be used for determining positional placement of annotations, and, by using multiple techniques, positioning can be made more precise and reliable. The level of detail of annotation information can be adjusted according to the precision of the techniques used. A trade-off can be taken into account between precision of annotation and communication cost, delay and/or power consumption. An annotation database can be updated in a self-organizing way. Public information as available on the web can be converted to annotation data.

Image annotation on portable devices

Techniques are shown for predicting the number of times a media selection will be consumed by one or more users at a target time. Examples of user behavior during the consumption of a media selection are chosen as input features. A partitioner separates a set of media selections into a training subset and an evaluation subset. The input features are transformed into feature vectors, and a learned function is derived to define a relationship between the feature vector for the training subset and the number of times a media selection from the training subset is consumed. The learned function is then applied to a feature vector for the evaluation subset to test its accuracy.

Prediction of media selection consumption using analysis of user behavior

Described herein are techniques related to prediction of video revenue for non-monetized videos. This Abstract is submitted with the understanding that it will not be used to interpret or limit the scope and meaning of the claims. A video-revenue prediction tool predicts revenue for non-monetized videos using historical revenue data of monetized videos.

Video-revenue prediction tool

A method of predicting interest levels associated with publication and content item combinations is described. Additionally, a server computing device for predicting interest levels associated with publication and content item combinations is described. Further, a computer-readable storage device having processor-executable instructions embodied thereon is described. The processor-executable instructions are for predicting interest levels associated with publication and content item combinations.

Predicting interest levels associated with publication and content item combinations

A system for creating audiences for a shared content publisher, includes a data store comprising a computer readable medium storing a program of instructions for audiences for a shared content publisher; a processor that executes the program of instructions; a data processor to monitor access to an Internet web site by a first set of users for a reference time period; a window extraction module, based on a reference split period, to divide the monitored access into a vector X and a vector Y, wherein vector X is defined by accesses by the first set of users before the reference split period, and vector Y is defined by accesses by the first user after the reference split period; and a data analysis module to create a model based on the vector X and the vector Y, to evaluate the model based on a second set of users accessing content similar to vector X, to create a final model based on the evaluation, and to score a group of users associated with the shared content publisher based on the final model, the data processor monitors accesses by the first user by recording content or shared content identification and additional information items.

Automatic audience creation for shared content

A classifier training system trains classifiers for inferring the geographic locations of videos. A number of classifiers are provided, where each classifier corresponds to a particular location and is trained from a training set of videos that have been labeled as representing the location. In one embodiment, the training set is further restricted to those videos in which a landmark matching the location label is detected. The classifier training system extracts, from each of these videos, features that characterize the video, such as audiovisual features, text features, address features, landmark features, and category features. Based on these features, the classifier training system trains a location classifier for the corresponding location. Each of the location classifiers can be applied to videos without associated location labels to predict whether, or how strongly, the video represents the corresponding location. The prediction can be used for a variety of purposes, such as automatic labeling of videos with locations, presentation of location-specific advertisements in association with videos, and display of video data on relevant portions of an electronic map.

Learning to geolocate videos

Photographic images can be used to enhance three-dimensional (3D) virtual models of a physical location. In an embodiment, a method of generating a 3D scene geometry includes obtaining a first plurality of images and corresponding distance measurements for a first vehicle trajectory; obtaining a second plurality of images and corresponding distance measurements for a second vehicle trajectory, the second vehicle trajectory intersecting the first vehicle trajectory; registering a relative vehicle position and orientation for one or more segments of each of a first vehicle trajectory and a second vehicle trajectory; generating a three-dimensional geometry for each vehicle trajectory; mapping the three-dimensional geometries for each vehicle trajectory onto a common reference system based on the registering; and merging the three-dimensional geometries from both trajectories to generate a complete scene geometry.

Urban geometry estimation from laser measurements

A system and method generates super-resolution images and videos using motion-compensated low-resolution images and videos. An image is selected as a primary image from a plurality of low-resolution images and the rest of the low-resolution images are registered as secondary images with respect to the primary image. Each registered secondary image is transformed to a motion compensated image. A mask value for a pixel in each motion compensated image is estimated. The super-resolution image of the primary image is generated by combining the mask values and the motion compensated secondary images. Similarly, a low-resolution video is segmented into a plurality of video objects, each of which is represented by an alpha layer. A super-resolution frame of the segmented video object is generated. The super-resolution frames of each segmented video object are combined using the alpha layers to create a super-resolution frame of the resulting video.

Enhanced image and video super-resolution processing

A classifier training system learns classifiers for categories by combining data from a category-instance repository comprising relationships between categories and more specific instances of those categories with a set of video classifiers for different concepts. The category-instance repository is derived from the domain of textual documents, such as web pages, and the concept classifiers are derived from the domain of video. Taken together, the category-instance repository and the concept classifiers provide sufficient data for obtaining accurate classifiers for categories that encompass other lower-level concepts, where the categories and their classifiers may not be obtainable solely from the video domain.

Learning category classifiers for a video corpus

In previously known imaging devices as in still and motion cameras, for example, image sensor signal response typically is linear as a function of intensity of incident light. Desirably, however, akin to the response of the human eye, response is sought to be nonlinear and, more particularly, essentially logarithmic. Preferred nonlinearity is realized in image sensor devices of the invention upon severely limiting the number of pixel states, combined with clustering of pixels into what may be termed as super-pixels.

Image sensor having nonlinear response

Standard video footage even from a single video camera can be used to obtain, in an automated fashion, a stroboscope sequence of a sports event, for example. The sequence may be represented as a static images of a photographic nature, or by a video sequence in which camera motion remains present, in which case the video sequence can be rendered as a panning camera movement on a stroboscope picture or as an animated stroboscope sequence in which the moving object leaves a trailing trace of copies along its path. Multiple cameras can be used for an expanded field of view or for comparison of multiple sequences, for example.

Growing community of inventors

Zurich, Switzerland

Luciano Sbaiz