Gracenote, Inc.

Amanmeet Garg

Aneesh Vartakavi

Konstantinos Antonio Dimitriou

Andreas Schmidt

Sharmishtha Gupta

Lakshika Balasuriya

Joshua Ernest Morris

Gannon Gesiriech

In one aspect, an example method includes (i) obtaining fingerprint repetition data for a portion of video content, with the fingerprint repetition data including a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content; (ii) identifying the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content; (iii) determining that the portion of video content corresponds to a program specified in an electronic program guide using a time stamp of the portion of video content; and (iv) storing an indication of the portion of video content in a data file for the program.

Identifying and labeling segments within video content

Methods and systems are disclosed for generating general feature vectors (GFVs), each simultaneously constructed for separate tasks of image reconstruction and fingerprint-based image discrimination. The computing system may include machine-learning-based components configured for extracting GFVs from images, signal processing for both transmission and reception and recovery of the extracted GFVs, generating reconstructed images from the recovered GFVs, and discriminating between fingerprints generated from the recovered GFVs and query fingerprints generated from query GFVs. A set of training images may be received at the computing system. In each of one or more training iterations over the set of training images, the components may be jointly trained with each training image of the set by minimizing a joint loss function computed as a sum of losses due to signal processing and recovery, image reconstruction, and fingerprint discrimination. The trained components may be configured for runtime implementation among one or more computing devices.

Unified representation learning of media features for diverse tasks

Methods and systems for automated video segmentation are disclosed. A sequence of video frames having video segments of contextually-related sub-sequences may be received. Each frame may be labeled according to segment and segment class. A video graph may be constructed in which each node corresponds to a different frame, and each edge connects a different pair of nodes, and is associated with a time between video frames and a similarity metric of the connected frames. An artificial neural network (ANN) may be trained to predict both labels for the nodes and clusters of the nodes corresponding to predicted membership among the segments, using the video graph as input to the ANN, and ground-truth clusters of ground-truth labeled nodes. The ANN may be further trained to predict segment classes of the predicted clusters, using the segment classes as ground truths. The trained ANN may be configured for application runtime video sequences.

Automated video segmentation

In one aspect, a method includes receiving podcast content, generating a transcript of at least a portion of the podcast content, and parsing the podcast content to (i) identify audio segments within the podcast content, (ii) determine classifications for the audio segments, (iii) identify audio segment offsets, and (iv) identify sentence offsets. The method also includes based on the audio segments, the classifications, the audio segment offsets, and the sentence offsets, dividing the generated transcript into text sentences and, from among the text sentences of the divided transcript, selecting a group of text sentences for use in generating an audio summary of the podcast content. The method also includes based on timestamps at which the group of text sentences begin in the podcast content, combining portions of audio in the podcast content that correspond to the group of text sentences to generate an audio file representing the audio summary.

System and method for multi-modal podcast summarization

In one aspect, an example method includes (i) obtaining fingerprint repetition data for a portion of video content, with the fingerprint repetition data including a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content; (ii) identifying the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content; (iii) determining that the portion of video content corresponds to a program specified in an electronic program guide using a timestamp of the portion of video content; and (iv) storing an indication of the portion of video content in a data file for the program.

In one aspect, a method includes detecting a fingerprint match between query fingerprint data representing at least one audio segment within podcast content and reference fingerprint data representing known repetitive content within other podcast content, detecting a feature match between a set of audio features across multiple time-windows of the podcast content, and detecting a text match between at least one query text sentences from a transcript of the podcast content and reference text sentences, the reference text sentences comprising text sentences from the known repetitive content within the other podcast content. The method also includes responsive to the detections, generating sets of labels identifying potential repetitive content within the podcast content. The method also includes selecting, from the sets of labels, a consolidated set of labels identifying segments of repetitive content within the podcast content, and responsive to selecting the consolidated set of labels, performing an action.

Growing community of inventors

Sunnyvale, CA, United States of America

Amanmeet Garg