Amazon Technologies, Inc.

Verizon Digital Media Services Inc.

Hooman Mahyar

Arjun Cholkar

Vimal Bhat

Harshal Dilip Wanjari

Jatin Jain

Abhinav Misra

Amir Reza Khakpour

Ryan Barlow Dall

Robert J Peters

Derek Shiell

Muhammad Yahia

Abhinav Jain

Avijit Vajpayee

Rafael Soltanovich

Sushanta Das

Shiva Krishnamurthy

Nikhil Kajale

Andrew James McVeigh

Shai Ben Nun

Hani Suleiman

Roya Hosseini

Donghyeok Heo

Moussa El Chater

Domenic Rigoglioso

Steven David Prinz

Neha Aggarwal

Craig Critchley

Zhikang Zhang

Shubham Kansal

David John Ossim

Shivakumar Krishnamurthy

Udit Bhatia

Described are example systems and methods generally directed to systems and methods for a real-time content recommendation service employing active guidance to facilitate efficient and effective recommendations of content to a user. A plurality of characteristic information associated with content items from a corpus of content items can be obtained and used to generate a content graph and a discovery map/decision tree. The discovery maps/decision trees may include content items and/or segments of content items that are arranged and connected in the discovery maps/decision trees based on their associated characteristic information to facilitate guided recommendations of content based on user interactions. The discovery map/decision tree may then be accessed to provide guided recommendations of content.

Actively guided content recommendation

Systems, methods, and computer-readable media are disclosed for systems and methods for automated identification and mapping of objects in video content. Example methods may include determining a first set of frames in video content, determining, using one or more object recognition algorithms, a first object present in the first set of frames, determining that a first product corresponding to the first object is present in a product catalog comprising a set of product images, associating a first product identifier of the first product with a video identifier of the video content, and causing presentation of a set of product identifiers associated with the video identifier.

Automated indication and mapping of objects in video content

Techniques for using a machine learning model to determine a proper subset of a multimedia file for a viewer based on their interest without the need to actively control a media player timeline are described. As one example, a computer-implemented method includes receiving a request at a content delivery service from a media player of a viewer to play a proper subset of a live multimedia file for a category of content of the live multimedia file without the viewer actively controlling a timeline of the media player of the viewer, determining an indication of a prior multimedia playing interaction of the viewer with the content delivery service, partitioning, by the content delivery service, the live multimedia file into a video portion, an audio portion, and a text portion, determining, by the content delivery service, one or more labels for the video portion, the audio portion, and the text portion, determining, by a machine learning model of the content delivery service, a proper subset of segments of the live multimedia file to send to the viewer based at least in part on the indication and the one or more labels, and live streaming the proper subset of segments of the live multimedia file to the media player of the viewer.

Machine learning model to determine a proper subset of a multimedia file for a viewer based on their interest

Systems, methods, and computer-readable media are disclosed for systems and methods for semantic video segmentation to identify objects appearing in video content. Example methods may include determining a voice-based request at a first device to identify an object appearing in video content, determining a first video identifier for video content being consumed in association with the first device, determining a timestamp associated with the video content at a time the voice-based request was received, determining, using one or more knowledge graphs, a set of product identifiers for products present in the video content at a scene corresponding to the timestamp, and generating a response to the voice-based request comprising a first product identifier in the set of product identifiers.

Semantic video segmentation to identify objects appearing in video content

Techniques are described for detecting desynchronization between an audio component and a video component of a media presentation. Feature sets may be determined for portions of the audio component and portions of the video component, which may then be used to generate correlations between portions of the audio component and portions of the video component. Synchronization may then be assessed based on the correlations.

Event based audio-video sync detection

This disclosure is directed to a system and method that detects duplicated content and/or media items. A media item can be split into media item portions. Based on the media item portions, features can be determined. Using the features, media item portion signatures can be determined to generate a media item signature. The media item signature can be compared with a different media item signature to determine duplicated content within the media items.

Automated detection of duplicate content in media items

Systems and methods are provided for detecting and correcting synchronization errors in multimedia content comprising a video stream and a non-original audio stream. Techniques for directly detecting synchronization of video and audio streams may be inadequate to detect synchronize errors for non-original audio streams, particularly where such non-original audio streams contain audio not reflective of events within the video stream, such as speaking dialog in a different language than the speakers of the video stream. To overcome this problem, the present disclosure enables synchronization of a non-original audio stream to another audio stream, such as an original audio stream, that is synchronized to the video stream. By comparison of signatures, the non-original and other audio stream are aligned to determine an offset that can be used to synchronize the non-original audio stream to the video stream.

Audio-video synchronization for non-original audio tracks

Intelligent reframing techniques are described in which content (e.g., a movie) can be generated in a different aspect ratio than previously provided. These techniques include obtaining various video frames having a first aspect ratio. Various objects can be identified within the frames. An object having the highest degree of importance in a frame can be selected and a focal point can be calculated based at least in part on that object. A modified version of the content can be generated in a second aspect ratio that is different from the first aspect ratio. The modified version can be generated using the focal point calculated based on the object having the greatest degree of importance. Using these techniques, the content can be provided in a different aspect ratio while ensuring that the most important features of the frame still appear in the new version of the content.

Intelligent reframing

A system can be configured to receive entertainment content requested by a user and identify content segments and content features from the entertainment content. The content segments can be utilized to identify portions of the entertainment content for enrichment and/or enhancement by the system. The content features can be utilized to associate the entertainment content and the content segments with supplemental content that includes or is associated with the content features. The content features can indicate genres, scene classifications, significant figures credited with creating the entertainment content, and other points of interests for users interested in the entertainment content. The associations between the entertainment content and the supplemental content can enable the system to engage the users by presenting the supplemental content determined to match interests of the users.

Interactive entertainment content

This disclosure is directed to a system and method that automatically detects repeated content within multiple media items. Content providers often include content, such as an introduction, near the beginning of a media item. In some circumstances, such as in the case of a series of television episodes, the content providers use the same content in each episode of the series. By dividing the media items into portions and analyzing the portions, the systems and methods described can automatically detect the repeated content. Using the detection of the repeated content, a user interface can then allow a user to bypass the repeated content during playback.

Automated detection of repeated content within a media series

Automated identification and mapping of objects in video content

Technologies are provided for the detection and recognition of overlaid content within video content. Some embodiments include a computing system that can receive data defining a sequence of frames corresponding to video content. The sequence of frames spans a defined time interval. The computing system can determine image changes between contiguous images defined by contiguous frames in the sequence of frames. A subset of the image changes can indicate static content within the video content, and another subset of the image changes can indicate non-static content. The computing system can then generate a composite image using at least the image changes, where the composite image includes an area representing the static content. Using the composite image, the computing system can classify the area as a defined visual element. Examples of the defined visual element include a logo and text.

Detection and recognition of overlaid content within video content

Systems, methods, and computer-readable media are disclosed for systems and methods for automated selection of color palettes for video content. Example methods may include determining, by one or more computer processors coupled to memory, a first segment of video content, the first segment comprising a first set of frames, determining, using a first video processing algorithm, a first object that is present in the first set of frames, and determining, using a second video processing algorithm, a first semantic characteristic of the first segment. Some example methods may include generating a first vector representing the first object and the first semantic characteristic, and generating, using a first neural network and the first vector, a first color palette recommendation for the first segment. Selection of the first color palette recommendation may cause a color filter to be applied to the first set of frames.

Automated selection of color palettes for video content using artificial intelligence

Systems, methods, and computer-readable media are disclosed for systems and methods for intelligent content rating determination. Example methods include determining presence of a first feature in a first frame of a video using an object recognition algorithm, determining presence of a second feature in an audio file associated with the video using an audio processing algorithm, and determining presence of a third feature in a text file associated with the video using a natural language processing algorithm. Certain embodiments may include generating a predicted content rating for the video using a machine learning model, where the predicted content rating is based at least in part on the first feature, the second feature, and the third feature, and using feedback data for the predicted content rating to retrain the machine learning model.

Growing community of inventors

Los Angeles, CA, United States of America

Hooman Mahyar