The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Nov. 10, 2020
Filed:
Aug. 29, 2018
International Business Machines Corporation, Armonk, NY (US);
International Business Machines Corporation, Armonk, NY (US);
Abstract
Embodiments of the present systems and methods may provide techniques for handling acoustic-lips timing mismatch in audiovisual processing. In embodiments, the context-dependent time shift between the audio and visual streams may be explicitly modeled using an attention mechanism. For example, in an embodiment, a computer-implemented method for determining a context-dependent time shift of audio and video features in an audiovisual stream or file may comprise receiving audio information and video information of the audiovisual stream or file, processing the audio information and video information separately to generate a new representation of the audio information, including information relating to features of the audio information, and a new representation of the video information, including information relating to features of the video information, and mapping features of the audio information and features of the video information using an attention mechanism to identify synchronized pairs of audio and video features.