Amazon Technologies, Inc.

Prabhakar Gupta

Kumar Keshav

Mayank Sharma

Honey Gupta

Anil Kumar Nelakanti

Shaktisingh P Shekhawat

Muhammad Raffay Hamid

Xiaohan Nie

Shixing Chen

Abhinav Aggarwal

Dongqing Zhang

Saravanan Santhamoorthy Theckyam

Palanivelu Balakrishnan

Anil Kumar Kumar Nelakanti

Systems, devices, and methods are provided for searchability and discoverability of contextually relevant frames within digital content. Digital content, such as videos, may be segmented to identify a plurality of shots. Discoverability may be performed by identifying key frames of the digital content and using a contrastive language-image pre-training (CLIP) model to determine contextual relevance of a frame or shot to textual information associated with the digital content. Searchability may be performed by receiving search parameters and applying various filters to digital content to identify frames or shots that satisfy a user's search query.

Searchability and discoverability of contextually relevant frames within digital content

Methods and apparatus are described for evaluating dubbing of media content. Phonemes in dubbed audio are extracted and mapped to visemes. Lip poses in video frames of the media content corresponding to the phonemes of the dubbed audio are compared to the visemes determined from the dubbed audio. A notification may be generated based on the comparison that indicates synchronization of the dubbed audio to lip poses of the video.

Audio-lip movement correlation measurement for dubbed content

Techniques for a computer-implemented service that utilizes a machine learning model to identify the salient and/or non-salient regions in a video frame are described. According to some embodiments, a computer-implemented method includes receiving a frame of a video at a content delivery service, generating, by a machine learning model of the content delivery service, a per pixel salience score map for the frame, and inserting, by the content delivery service, secondary content into the frame based at least in part on the per pixel salience score map for the frame.

Computer-implemented methods for determining salient and non-salient regions in video frames

Described herein is a computer-implemented method for extracting and identifying an audio song. An audio file can be accessed by a computing device. A set of audio categories and a set of probabilities associated with the set of audio categories can be determined for a first audio clip. A subset of the set of audio categories can be determined based on a subset of the set of probabilities. Each audio category of the subset of the set of audio categories can correspond to an audio class label. Whether the first audio clip is part of a song can be determined. The song can be defined by combining the first audio clip with other audio clips.

Song generation using a pre-trained audio neural network

A respective set of features, including emotion-related features, are extracted from segments of a video for which a preview is to be generated. A subset of the segments is chosen using the features and filtering criteria including at least one emotion-based filtering criterion. Respective weighted preview-suitability scores are assigned to the segments of the subset using at least a metric of similarity between individual segments and a plot summary of the video. The scores are used to select and combine segments to form a preview for the video.

Automated preview generation for video entertainment content

Technologies are provided for automated quality assessment of translations. In some embodiments, quality of a translation can be assessed by generating a machine-learning (ML) model that classifies the translation as pertaining to one of three quality categories. A first quality category can include, for example, translations that are deemed satisfactory. A second quality category can include, for example, translations that are deemed subject to edition prior to being deemed satisfactory. A third quality category can include, for example, translations that are deemed unsatisfactory. The generated ML model can then be applied to the translation and a corresponding sentence in a source language in order to classify the translation as pertaining to one of the three categories.

Automated quality assessment of translations

An automated solution to determine suitable time ranges or timestamps for captions is described. In one example, a content file includes subtitle data with captions for display over respective timeframes of video. Audio data is extracted from the video, and the audio data is compared against a sound threshold to identify auditory timeframes in which sound is above the threshold. The subtitle data is also parsed to identify subtitle-free timeframes in the video. A series of candidate time ranges is then identified based on overlapping ranges of the auditory timeframes and the subtitle-free timeframes. In some cases, one or more of the candidate time ranges can be merged together or omitted, and a final series of time ranges or timestamps for captions is obtained. The time ranges or timestamps can be used to add additional non-verbal and contextual captions and indicators, for example, or for other purposes.

Caption timestamp predictor

A context-aware spell checker to detect non-word spelling errors and/or suggest corrections. The context-aware spell checker may utilize n-gram conditional probabilities to suggest corrections based on a context of the non-word spelling error. The suggested corrections may be presented as a prioritized list of words based on calculated scores of the n-gram conditional probabilities. Utilizing n-gram conditional probabilities may permit the context-aware spell checker to be integrated across a multitude of languages or configured according to a particular language. The context-aware spell checker may perform spell checking and suggest corrections in real-time, or may be at least partially automated, to reduce user perceived latency and delay.

Context-aware spell checker

Disclosed are various embodiments for evaluating the accuracy of a translation of a source text. Word embeddings from a first language and a second language are aligned in a shared vector space. Word pairs from the sourced text and translated text are then identified. Subsequently, similarity scores between respective word embeddings for the words in the word pair are calculated. Word pairs are then selected based on the similarity scores. The accuracy of the translation is then evaluated based at least in part on the selected word pairs.

Growing community of inventors

Delhi, India

Prabhakar Gupta