Microsoft Technology Licensing, LLC

Yu-Fei Ma

Hong-Jiang Zhang

Lie Lu

Wei-Ying Ma

Xian-Sheng Hua

Mingjing Li

Bin Lin

Gu Xu

Zhike Kong

Xinli Zou

Dongjun Lan

Systems and methods to generate a motion attention model of a video data sequence are described. In one aspect, a motion saliency map B is generated to precisely indicate motion attention areas for each frame in the video data sequence. The motion saliency maps are each based on intensity I, spatial coherence Cs, and temporal coherence Ct values. These values are extracted from each block or pixel in motion fields that are extracted from the video data sequence. Brightness values of detected motion attention areas in each frame are accumulated to generate, with respect to time, the motion attention model.

Generating a motion attention model

Systems and methods for smart media content thumbnail extraction are described. In one aspect program metadata is generated from recorded video content. The program metadata includes one or more key-frames from one or more corresponding shots. An objectively representative key-frame is identified from among the key-frames as a function of shot duration and frequency of appearance of key-frame content across multiple shots. The objectively representative key-frame is an image frame representative of the recorded video content. A thumbnail is created from the objectively representative key-frame.

Systems and methods for smart media content thumbnail extraction

Systems and methods for image attention analysis are described. In one aspect, image attention is modeled by preprocessing an image to generate a quantized set of image blocks. A contrast-based saliency map for modeling one-to-three levels of image attention is then generated from the quantized image blocks.

Contrast-based image attention analysis framework

A robust camera motion analysis method is described. In an implementation, a method includes analyzing video having sequential frames to determine one or more camera motions that occurred when sequential frames of the video were captured. The one or more camera motions for each frame are described by a set of displacement curves, a mean absolute difference (MAD) curve, and a major motion (MAJ) curve. The set of displacement curves describe the one or more camera motions in respective horizontal (H), vertical (V), and radial (R) directions. The MAD curve relates a minimum MAD value from the set of displacement curves. The MAJ curve is generated from the minimum MAD value and provides one or more qualitative descriptions that describe the one or more camera motions as at least one of still, vertical, horizontal and radial.

Robust camera motion analysis for home video

Systems and methods for representing sequential motion patterns are described. In one aspect, video frames are converted into a sequence of energy redistribution measurements. One or more motion filters are then applied to the ER measurements to generate one or more temporal sequences of motion patterns, the number of temporal sequences being a function of the number of motion filters.

Sequential motion pattern representation

Systems and methods to generate an attention model for computational analysis of video data are described. In one aspect, feature components from a video data sequence are extracted. Attention data is generated by applying multiple attention models to the extracted feature components. The generated attention data is integrated into a comprehensive user attention model for the computational analysis of the video data sequence.

Systems and methods for generating a comprehensive user attention model

Systems and methods to automatically edit a video to generate a video summary are described. In one aspect, sub-shots are extracted from the video. Importance measures are calculated for at least a portion of the extracted sub-shots. Respective relative distributions for sub-shots having relatively higher importance measures as compared to importance measures of other sub-shots are determined. Based on the determined relative distributions, sub-shots that do not exhibit a uniform distribution with respect to other sub-shots in the particular ones are dropped. The remaining sub-shots are connected with respective transitions to generate the video summary.

Growing community of inventors

Beijing, China

Yu-Fei Ma