Amazon Technologies, Inc.

Daniel Stephen ReMine

Rohith Mysore Vijaya Kumar

Najmeh Sadoughi Nourabadi

Abhishek Yanamandra

Chengyuan Liu

Techniques for performing a machine learning (ML) cinematic (e.g., movie) question answering are described. According to some examples, a computer-implemented method includes receiving a request from a viewer device at a content delivery service to play a video; sending the video from the content delivery service to the viewer device; receiving, by the content delivery service, a question from the viewer device during playing of the video; generating, by a script context retrieval machine learning model of the content delivery service, a proper subset of a script of the video based on an input of the question; generating, by a cinematic question answering machine learning model of the content delivery service, an answer based on an input of the proper subset of the script of the video; and sending, by the content delivery service, the answer to the viewer device.

Computer-implemented methods for movie question answering as a second screen experience using a large language machine learning model

Visual content of media programs is recognized using text-based scripts of the media programs. A script of a media program includes sets of words to be spoken by actors during the media program, and also descriptions of points of interest of the media program. A file of subtitles or captions of a media program includes sets of words actually spoken by actors during the media program along with marks or stamps of times when such words were spoken. The sets of words of a script and in subtitles or captions are processed to determine where such words align. The sets of words of the script are then marked or stamped with times from the subtitles. The descriptions may then be processed to predict the visual content from such descriptions, and times at which the visual content appears within the media program are determined from the sets of words.

Growing community of inventors

Seattle, WA, United States of America

Daniel Stephen ReMine