Princeton University

Mitsubishi Electric Research Laboratories, Inc.

General Electric Company

United States of America As Represented by the Secretary of the Air Force

Mitsubishi Electric Research Labs.

Mitsubishi Denki Kabushiki Kaisha

Bede Liu

Anthony Vetro

Huifang Sun

Peng Yin

Wayne A Wolf

Tommy C Poon

Boon-Lock Yeo

Minerva M Yeung

Andre Zaccarin

Stuart C Schwartz

Min Wu

Juhua Zhu

Minghui Xia

Abraham Peled

Richard P Perry

Veronika M Monteleone

The present invention relates to a method and system for foreground segmentation in which frames of a video sequence are analyzed in the transform domain to determine one or more features. The features are used to model the background. The background can be modeled as a single Gaussian model with a mean and variance of the features. A current frame is segmented by determining if one or more features of the current frame analyzed in the foreground domain satisfy a threshold between the background model. The threshold value can be based on the mean and/or variance of features. During the segmentation, the mean and variance can be updated based on previous corresponding values and current features to adaptively update the background model. In one embodiment, the frames are divided into a plurality of blocks. A transform is used to analyze the blocks in the transform domain. For example, the transform can be a discrete cosine transform (DCT). The features can be a DC feature comprising the DC coefficient and an AC feature comprising a weighted sum of the AC coefficients. The weighted sum is determined with weights which are varied to emphasize different aspects of the present invention. Additional processing steps can be used to remove false positives, handle sudden global illumination changes, handle sudden local illumination changes and remove false negatives.

Method and apparatus for foreground segmentation of video sequences

A model stored in a memory accessible by a video transcoder includes a first rate-distortion function modeling a requantization of an input video. A second-rate distortion function models a resynchronization marker insertion rate for the transcoded video, and a third rate-distortion function models an intra-block insertion rate for the transcoded video.

Rate-distortion models for error resilient video transcoding

A method and system reduces the spatial resolution of a compressed bitstream of a sequence of frames of a video signal by first decoding the frames, and storing the decoded frames in a first frame buffer. While performing the decoding, motion compensating is performed with full resolution motion vectors of the stored decoded frames. The decoded frames are then down-sampled to a reduced resolution, and stored in a second frame buffer. The reduced resolution frames are partially encoded to produce a reduced resolution compressed bitstream of the video. While performing the partial encoding, motion compensation is performed with reduced resolution motion vectors of the stored reduced resolution frames.

Video transcoder with spatial resolution reduction and drift compensation

A method compensates for drift in macroblocks of a partially decoded input bitstream. The macroblocks include intra-mode and inter-mode macroblocks, and each macroblock includes DCT coefficients, and at least one motion vector. An estimate of drift is measured in the partially decoded input bitstream. The estimated drift is translated into an intra refresh rate. The modes of inter-mode macroblock are mapped to intra-mode macroblock according to the refresh rate. The DCT coefficients and the motion vector for each changed macroblock are modified in accordance with the mapping for each changed macroblock.

Video transcoder with drift compensation

A method transcodes groups of macroblocks of a partially decoded input bitstream. The groups of macroblocks include intra-mode and inter-mode macroblocks. Each macroblock includes DCT coefficients, and at least one motion vector. The modes of each group of macroblocks are mapped to be identical only if there is an inter-mode block and an intra-mode macroblock in the group. If any of the macroblocks in the group are mapped, then the DCT coefficients and the motion vector for such mapped macroblocks are modified in accordance with the mapping to generate reduced-resolution macroblock for an output compressed bitstream to compensate for drift.

Video transcoder with spatial resolution reduction

A digital watermarking process whereby an invisible watermark inserted into a host image is utilized to determine whether or not the image has been altered and, if so, where in the image such alteration occurred. The watermarking method includes the steps of providing a look-up table containing a plurality of coefficients and corresponding values; transforming the image into a plurality of blocks, wherein each block contains coefficients matching coefficients in the look-up table; and embedding the watermark in the image by performing the following substeps for at least some of the blocks: First, a coefficient is selected for insertion of a marking value representative of a corresponding portion of the watermark. Next, the value of the selected coefficient to used to identify a corresponding value in the look-up table. Finally, the identified coefficient is left unchanged if the corresponding value is the same as the marking value, and is changed if the corresponding value is different from the marking value. After the insertion of the watermark, the image may be stored in a lossy-compression form, thus permitting efficient storage and distribution. Moreover, the method may be used to produce two output signals for authentication: (1) a meaningful pattern to facilitate a quick visual check, and (2) an additional signal to detect unauthorized alteration. The method can be applied to an image compressed using JPEG or other techniques, such as Wavelet compression, and the marked image can be kept in the compressed format. Any alteration made on the marked image can be localized, making the method suitable for use in a &ldquo;trustworthy&rdquo; digital camera or camcorder.

Watermarking scheme for image authentication

A new technique for extracting a hierarchical decomposition of a complex video selection for browsing purposes, combines visual and temporal information to capture the important relations within a scene and between scenes in a video, thus allowing the analysis of the underlying story structure with no a priori knowledge of the content. A general model of hierarchical scene transition graph is applied to an implementation for browsing. Video shots are first identified and a collection of key frames is used to represent each video segment. These collections are then classified according to gross visual information. A platform is built on which the video is presented as directed graphs to the user, with each category of video shots represented by a node and each edge denoting a temporal relationship between categories. The analysis and processing of video is carried out directly on the compressed videos. Preliminary tests show that the narrative structure of a video selection can be effectively captured using this technique.

Method and apparatus for video browsing based on content and structure

In an image coding system, block based motion estimation is used to determine motion vectors for blocks of pixels in a current frame or field. Only a portion of blocks from a predetermined pattern of blocks in a current frame or field are searched for a match with a block of pixels in a previous frame or field over a designated search area. Motion vectors for the blocks of the current frame or field not selected for searching are obtained by interpolation from motion vectors obtained for associated searched neighboring blocks, respectively, or by using the zero motion vector, or by performing a search over a limited range around the position defined by each of the motion vectors from neighboring blocks.

Method and apparatus for determining motion vectors for image sequences

In an imaging system, block based motion estimation is used to determine motion vectors for blocks of pixels in a current frame or field. Only a portion of blocks from a predetermined pattern of blocks in a current frame or field are searched for a match with a block of pixels in a previous frame or field over a designated search area. Motion vectors for the blocks of the current frame or field not selected for searching are obtained by interpolation from motion vectors obtained for associated searched neighboring blocks, respectively.

The detector is responsive to a modulated intermediate frequency (IF) or radio frequency (RF) signal for producing four sample signals spaced at 90.degree. one-quarter IF (or RF) cycle apart, the first and third sample signals corresponding to the amplitude of the IF signal and its negative value, respectively, the stream of such sample signals corresponding to the inphase component of the IF signal; the fourth and second sample signals corresponding to the amplitude of the IF signal and its negative value, respectively, the stream of such sample signals corresponding to the quadrature component of the IF signal.

Digital inphase/quadrature product detector

A combinatorial digital filter apparatus utilizing a second order filter in which bits are processed simultaneously rather than serially.

Growing community of inventors

Princeton, NJ, United States of America

Bede Liu