Mitsubishi Electric Research Laboratories, Inc.

Kadir A Peker

Ajay Divakaran

Huifang Sun

Regunathan Radhakrishnan

Daniel Nikolaev Nikovski

Ziyou Xiong

Koji Miyahara

Romain Cabasson

A method plays back a video at speeds adapted to content of the video. A video is partitioned into summary segments and skipped segments. The summary segments are played back sequentially at a normal play back speed, and the skipped segments are played back at varying speeds corresponding to a visual complexity of the skipped segments.

Method and system for playing back videos at speeds adapted to content

A method generates a summary of a video. Faces are detected in a plurality of frames of the video. The frames are classified according to a number of faces detected in each frame and the video is partitioned into segments according to the classifications to produce a summary of the video. For each frame classified as having a single detected face, one or more characteristics of the face is determined. The frames are labeled according to the characteristics to produce labeled clusters and the segments are partitioned into sub-segments according to the labeled clusters.

Method and system for segmenting videos using face detection

A method and system detects and diagnoses faults in heating, ventilating and air conditioning (HVAC) equipment. Internal state variables of the HVAC equipment are measured under external driving conditions. Expected internal state variables are predicted for the HVAC equipment operating under the external driving conditions using a locally weighted regression model. Features are determined of the HVAC based on differences between the measured and predicted state variables. The features are classified to determine a condition of the HVAC equipment.

Detecting and diagnosing faults in HVAC equipment

A method plays frames of a video adaptively according to a visual complexity of the video. First a spatial frequency of pixel within frames of the video is measured, as well as a temporal velocity of corresponding pixels between frames of the video. The spatial frequency is multiplied by the temporal velocity to obtain a measure of the visual complexity of the frames of the video. The frames of the video are then played at a speed that corresponds to the visual complexity.

Visual complexity measure for playing videos adaptively

A method mines unknown content of a video by first selecting one or more low-level features of the video. For each selected feature, or combination of features, time series data is generated. The time series data is then self-correlated to identify similar segments of the video according to the low-level features. The similar segments are grouped into clusters to discover high-level patterns in the unknown content of video.

Video mining using unsupervised clustering of video content

A method detects short term, unusual events in a video. First, features are extracted features from the audio and the video portions of the video. Segments of the video are labeled according to the features. A global sliding window is applied to the labeled segments to determine global characteristics over time, while a local sliding window is applied only to the labeled segments of the global sliding window to determine local characteristic over time. The local window is substantially shorter in time than the global window. A distance between the global and local characteristic is measured to determine occurrences of the unusual short time events.

Method for detecting short term unusual events in videos

A method summarizes unknown content of a video. First, low-level features of the video are selected. The video is then partitioned into segments according to the low-level features. The segments are grouped into disjoint clusters where each cluster contains similar segments. The clusters are labeled according to the low-level features, and parameters characterizing the clusters are assigned. High-level patterns among the labels are found, and the these patterns are used to extract frames from the video according to form a content-adaptive summary of the unknown content of the video.

Blind summarization of video content

A method measures an intensity of motion activity in a compressed video. The intensity of the motion activity is used to partition the video into segments of equal cumulative motion activity. Key-frames are then selected from each segments. The selected key-frames are concatenated in temporal order to form a summary of the video.

Method for summarizing a video using motion descriptors

A method discovers patterns in unknown multi-dimensional data. A time-series of the multi-dimensional data is generated and a point cross-distance matrix is constructed by self-correlating the time-series. All minimum cost paths in the point cross-distance matrix are located at multiple time resolutions. The minimum cost paths are then related to temporal sub-sequences in the multi-dimensional data to discover high-level patterns in the unknown multi-dimensional data.

Pattern discovery in multi-dimensional time series using multi-resolution matching

A system and method for temporally processing an input video including input frames. Each frame has an associated frame play time, and the input video has a total input video play time that is a sum of the input frame play times of all of the input frames. Each of the input frames is classified according to a content characteristic of each frames. An output frame play time is allocated to each of the input frames that is based on the classified content characteristic of each of the input frames to generate a plurality of output frames that form an output video.

Adaptively processing a video based on content characteristics of frames in a video

A method for summarizing a video first detects audio peaks in a sub-sampled audio signal of the video. Then, motion activity in the video is extracted and filtered. The filtered motion activity is quantized to a continuous stream of digital pulses, one pulse for each frame. If the motion activity is greater than a predetermined threshold the pulse is one, otherwise the pulse is zero. Each quantized pulse is tested with respect to the timing of rising and falling edges. If the pulse meets the condition of the test, then the pulse is selected as a candidate pulse related to an interesting event in the video, otherwise the pulse is discarded. The candidate pulses are correlated, time-wise to the audio peaks, and patterns between the pulses and peaks are examined. The correlation patterns segment the video into uninteresting and interesting portions, which can then be summarized.

Summarizing videos using motion activity descriptors correlated with audio features

A method extracts an intensity of motion activity from shots in a compressed video. The method then uses the intensity of motion activity to segment the video into easy and difficult segments to summarize. Easy to summarize segments are represented by any frames selected from the easy to summarize segments, while a color based summarization process extracts generates sequences of frames from each difficult to summarize segment. The selected and generated frames of each segment in each shot are combined to form the summary of the compressed video.

Method for summarizing a video using motion and color descriptors

A method describes motion activity in a video sequence. A motion activity matrix is determined for the video sequence. A threshold for the motion activity matrix is determined. Connected regions of motion vectors at least equal to the threshold are identified and measured for size. A histogram of the distribution of the sizes of the connected areas is constructed for the entire video sequence. The histogram is normalized to characterize the spatial distribution of the video sequence in a motion activity descriptor.

Growing community of inventors

North Arlington, NJ, United States of America

Kadir A Peker