Microsoft Technology Licensing, LLC

Mattan Serry

Zvi Figov

Yonit Hoffman

Maayan Yedidia

Irit Ofer

Eliyahu Strugo

Michael Ben-haym

Mordechai Kadosh

Systems, methods, and a computer-readable medium are provided for matching textless elements to texted elements in video content. A video processing system including a textless matching system may divide a video into shots, identify shots having similar durations, identify sequences of shots having similar durations, and compare image content in representative frames of the sequences to determine whether the sequences match. When the sequences are determined to match, the sequences may be paired, wherein the first sequence may include shots with overlaid text and the second sequence may include textless version of corresponding texted shots included in the first sequence. In some examples, the video processing system may further replace the determined corresponding texted shots.

Textless material scene matching in videos

Disclosed is a method for automatically detecting an introduction/opening song within a multimedia file. The method includes designating sequential blocks of time in the multimedia file as scene(s) and detecting certain feature(s) associated with each scene. The extracted scene feature(s) may be analyzed and used to assign a probability to each scene that the scene is part of the introduction/opening song. The probabilities may be used to classify each scene as either correlating to or not correlating to, the introduction/opening song. The temporal location of the opening song may be saved as index data associated with the multimedia file.

Combining visual and audio insights to detect opening scenes in multimedia files

An approach using 3D algorithms to solve 2D head localization problems is disclosed. A system can extrapolate aspects of one part of an object, e.g., extract characteristics of a person's head, using a 2D input image of another part of the object, e.g., a 2D image of the person's face. The system then selects an appropriate 3D model by the use of facial features detected in an image of a person's face. Using the selected 3D model and the 3D rotation angles provided by a face detector, the system rotates the model and then projects the model to a 2D shape. The system then scales and translates, e.g., transforms, the 2D shape to match the 2D face bounding box. Then, using the transformed 2D shape, the system extracts a bounding box for the extracted portion of an object, e.g., the head of the person depicted in the 2D input image.

Head position extrapolation based on a 3D model and image data

Growing community of inventors

Herzliya, Israel

Mattan Serry