Netflix Inc.

Kyle Tacke

Yadong Wang

Shilpa Jois Rao

Andrew Kerr Swan

Raja Ranjan Senapati

Murthy Parthasarathi

Chih-Wei Wu

Boney Sekh

Anjali Chablani

The disclosed computer-implemented method may include receiving, as input, an audio/video data object; isolating a video stream of a visible potential speaker over a plurality of frames of the audio/video data object; isolating an audio stream over the plurality of frames; providing the isolated video stream and the isolated audio stream to a machine learning model trained with contrastive learning, the contrastive learning using (i) a corpus of video segments of visible speakers with corresponding original audio for positive samples; and (ii) a corpus of video segments of visible speakers with corresponding dubbed audio for negative samples; and evaluating a match between the isolated audio stream and the isolated video stream based at least in part on an output of the machine learning model. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for active speaker detection

The disclosed computer-implemented method may include (1) accessing a first media data object and a different, second media data object that, when played back, each render temporally sequenced content, (2) comparing first temporally sequenced content represented by the first media data object with second temporally sequenced content represented by the second media data object to identify a set of common temporal subsequences between the first media data object and the second media data object, (3) identifying a set of edits relative to the set of common temporal subsequences that describe a difference between the temporally sequenced content of the first media data object and the temporally sequenced content of the second media data object, and (4) executing a workflow relating to the first media data object and/or the second media data object based on the set of edits. Various other methods, systems, and computer-readable media are also disclosed.

Automated workflows from media asset differentials

The disclosed computer-implemented method may include accessing an audio track that is associated with a video recording, identifying a section of the accessed audio track having a specific audio characteristic, reducing a volume level of the audio track in the identified section, accessing an audio segment that includes a synthesized voice and inserting the accessed audio segment into the identified section of the audio track, where the inserted segment has a higher volume level than the reduced volume level of the audio track in the identified section. The synthesized voice description can be used to provide additional information to a visually impaired viewer without interrupting the audio track that is associated with the video recording, typically by inserting the synthesized voice description into a segment of the audio track in which there is no dialog. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for mixing synthetic voice with original audio tracks

The disclosed computer-implemented method may include obtaining an audio sample from a content source, inputting the obtained audio sample into a trained machine learning model, obtaining the output of the trained machine learning model, wherein the output is a profile of an environment in which the input audio sample was recorded, obtaining an acoustic impulse response corresponding to the profile of the environment in which the input audio sample was recorded, obtaining a second audio sample, processing the obtained acoustic impulse response with the second audio sample, and inserting a result of processing the obtained acoustic impulse response and the second audio sample into an audio track. Various other methods, systems, and computer-readable media are also disclosed.

Growing community of inventors

Glendale, CA, United States of America

Kyle Tacke