Google Inc.

Min-Hsuan Tsai

Sudheendra Vijayanarasimhan

Sanketh Shetty

Balakrishnan Varadarajan

Weilong Yang

Tomas Izo

George Dan Toderici

Nitin Khandelwal

Apostol Ivanov Natsev

Nicola Muscettola

Sami Ahmad Abu-El-Haija

WeiHsin Gu

Ramesh Rangarajan Sarukkai

Chenjie Gu

Jun Hong

Hassan Akbari

Jingbin Wang

Pei Cao

Phuong B Le

Yun Shi

Ji Zhang

Jianfeng Yang

Susanna Maria Ricco

Wei-Hong Chuang

Honglu Zhou

Zheng Sun

Carolyn F Au

Tianpeng Jin

Xiangrong Chen

Yi Shen

Andrei Kapishnikov

James Futrell

Susana Ricco

Marco Ziccardi

Rahul Sunil Bhalerao

Ye Xia

Madhuri Shanbhogue

Mojtaba Seyedhosseini

A method includes obtaining first data including a first identifier of a first product determine in association with a content item based on first metadata of the content item. The method further includes obtaining a first confidence value associated with the first product and the content item. The method further includes obtaining second data including a second identifier of the first product and a second confidence value. The method further includes providing the first data and the second data to a trained machine learning model. The method further includes obtaining a third confidence value from the trained machine learning model associated with the first product. The method further includes adjusting second metadata of the content item in view of the third confidence value.

Product identification in media items

Methods and systems for time marking of media items at a platform using machine-learning are provided herein. An indication of a identified media item is provided as input to a machine-learning model and one or more outputs of the machine-learning model is obtained. The one or more obtained outputs comprise time marks identifying each of the plurality of content segments of the media item. Each of the plurality of content segments is associated with a segment start indicator for a timeline of the media item. A resulting duration is determined of a combination of the plurality of content segments for which the time marks were obtained from the one or more of outputs of the machine-learning model. Responsive to determining that the resulting duration is less than the duration of the media item, one or more further inputs is provided to the machine learning model.

Time marking chapters in media items at a platform using machine-learning

Systems and methods of automatically extracting summaries of video content are described herein. A data processing system can access, from a video database, a first video content element including a first plurality of frame. The data processing system can select an intervallic subset of the first plurality of frames of the first video content element. The data processing system can calculate, for each of a plurality of further subsets comprising a predetermined number of frames from the intervallic subset, a score for the further subset. The data processing system can identify, from the plurality of further subsets, a further subset having a highest score. The data processing system can select a portion of the first video content element comprising the frames of the further subset having the highest score. The data processing system can generate a second video content element comprising the selected portion of the first video content element.

Summarizing video content

A computer-implemented method for selecting representative frames for videos is provided. The method includes receiving a video and identifying a set of features for each of the frames of the video. The features including frame-based features and semantic features. The semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video. A set of video segments for the video is subsequently generated. Each video segment includes a chronological subset of frames from the video and each frame is associated with at least one of the semantic features. The method generates a score for each frame of the subset of frames for each video segment based at least on the semantic features, and selecting a representative frame for each video segment based on the scores of the frames in the video segment. The representative frame represents and summarizes the video segment.

Selecting and presenting representative frames for video previews

Methods and systems for time marking of media items at a platform using machine-learning are provided herein. A media item is provided to users of a platform. An indication of the identified media item is provided as input to a machine-learning model that is trained using different feature types of historical media items to predict a plurality of content segments of a given media item each depicting, to the one or more users, a distinct section of the media item. One or more outputs of the machine-learning model are obtained comprising time marks identifying each of the plurality of content segments of the media item. Each of the plurality of content segments are associated with a segment start indicator for a timeline of the media item. The media item and an indication of each segment start indicator is provided for presentation to at least one user.

Systems and methods are disclosed for identifying transitions within media content items. In one implementation, a processing device process a first media content item, the first media content item being associated with a transition, to identify one or more characteristics associated with the transition. The processing device processes a second media content item to identify at least one of the one or more characteristics at a chronological interval of the second media content item. The processing device receives a sponsored content item. The processing device provides, during a presentation of the second media content item, the sponsored content item at the chronological interval.

Identifying transitions within media content items

A solution is provided for temporally segmenting a video based on analysis of entities identified in the video frames of the video. The video is decoded into multiple video frames and multiple video frames are selected for annotation. The annotation process identifies entities present in a sample video frame and each identified entity has a timestamp and confidence score indicating the likelihood that the entity is accurately identified. For each identified entity, a time series comprising of timestamps and corresponding confidence scores is generated and smoothed to reduce annotation noise. One or more segments containing an entity over the length of the video are obtained by detecting boundaries of the segments in the time series of the entity. From the individual temporal segmentation for each identified entity in the video, an overall temporal segmentation for the video is generated, where the overall temporal segmentation reflects the semantics of the video.

Entity based temporal segmentation of video streams

Growing community of inventors

Grapevine, TX, United States of America

Min-Hsuan Tsai