Seiko Epson Corporation

William Chen

Vasudev Bhaskaran

Changick Kim

Norihisa Hagiwara

Jau-Yuen Chen

An inter mode for encoding a video macroblock is selected. An initial analysis on the macroblock homogeneity is performed to quickly reduce the number of inter modes for which motion estimation and rate distortion calculation are to be performed. An inter mode is selected based on these calculations and sub-pixel motion estimation, at various sub-pixel interpolation granularities, is performed only on the selected inter mode. Sub-pixel motion estimation may be skipped for macroblocks that exhibit relatively low distortion at the integer pixel level.

Macroblock homogeneity analysis and inter mode prediction

Transform based distortion cost estimation, which may be used to calculate a rate distortion cost associated with a mode for encoding a macroblock, is described. In one embodiment of the invention, a distortion value for a particular mode is estimated within the transform domain, which allows for the elimination of both inverse transformation and inverse motion functions in this calculation. A spatial domain residual of an encoding mode is estimated by identifying a difference, within the transform domain, between a motion compensated prediction residual of a macroblock and its corresponding reconstructed signal. The estimated spatial domain residual may then be used in distortion matrix computations to estimate a distortion level, within the transform domain, for an encoding mode.

Transform domain based distortion cost estimation

The temporal and/or spatial characteristics of a macroblock are analyzed in order to reduce the number of modes for which motion estimation and rate distortion efficiency calculations are to be performed. In one embodiment, macroblock mean and variance characteristics are analyzed to merge sub-blocks together within the macroblock. These merged sub-blocks may be used to identify both inter and intra modes for the macroblock.

Temporal and spatial analysis of a video macroblock

An inter-mode for encoding a video macroblock is selected based on a comparison of at least two rate-distortion values associated with inter-modes. The number of potential inter-modes for encoding the video macroblock is initially reduced based on the rate-distortion calculation of the SKIP mode between the current macroblock and the collocated macroblock (corresponding to a 0,0 motion vector). Motion estimation is performed on the remaining inter-modes to identify reference blocks and motion vectors. The number of calculations performed in identifying the reference blocks and motion vectors is potentially reduced because associations between inter-modes are recognized and leveraged.

Motion estimation and inter-mode prediction

A prediction mode for encoding data is selected using a reduced number of rate-distortion cost computations. This reduction in rate-distortion cost computations is caused by filtering the number of potential intra prediction modes based on two criteria. First, the number of potential prediction modes is reduced based on at least one characteristic of a quantization procedure performed during the encoding procedure. Second, the number of potential prediction modes is reduced based on an error value calculated for each of the potential prediction modes.

Prediction intra-mode selection in an encoder

A method for reducing the memory requirements for decoding a bit stream is provided. The method initiates with receiving a video bit stream. Then, a frame of the bit stream is decoded into a discrete cosine transform (DCT) domain representation. Next, non-zero coefficients of the DCT domain representation are identified. Then, a hybrid data structure is assembled. The hybrid data structure includes a fixed size array and a variable size overflow vector. Next, the non-zero coefficients of the DCT domain representation are inserted into the hybrid data structure. A computer readable media, a printed circuit board and a device configured to decode video data are also provided.

Method and apparatus for memory efficient compressed domain video processing

An interactive photo system that provides an interesting and entertaining way to link photos from the digital and analog domains. The system enables easy conversion of a paper-based photo into digital form. The conversion process generally involves a user holding up a paper-based photo image in front of a camera of the system, such that the captured image is rendered in a specific location on a display screen. The system adjusts the captured image for orientation and illumination irregularities in creating a digital version of the paper-based photo. The system further includes a database of digital images and a content retrieval engine to which content from the just-converted digital image can be used as a search query to search the database for other similar digital photos.

Interactive photo system

A method for creating a summary of an audiovisual presentation initiates with segmenting a frame of the audiovisual presentation. Then, a slide region of the segmented frame is identified. Next, a histogram representing lines in the slide region is generated. Next, moving regions associated with successive frames from the histogram are suppressed. A computer readable medium, a system for providing a real time summarization of a meeting and an integrated circuit are also provided.

Method and apparatus for summarizing and indexing the contents of an audio-visual presentation

A method for reducing the memory requirements for decoding a bit stream is provided. The method initiates with receiving a video bit stream. Then, a frame of the bit stream is decoded into a transform domain representation. Next, non-zero coefficients of the transform domain representation are identified. Then, a hybrid data structure is assembled. The hybrid data structure includes a fixed size array and a variable size overflow vector. Next, the non-zero coefficients of the transform domain representation are inserted into the hybrid data structure. A computer readable media, a printed circuit board and a device configured to decode video data are also provided.

Method and apparatus for transform domain video processing

A power-scalable hybrid technique to reduce blocking and ringing artifacts in low bit-rate block-based video coding is employed in connection with a modified decoder structure. Fast inverse motion compensation is applied directly in the compressed domain, so that the transform (e.g., DCT) coefficients of the current frame can be reconstructed from those of the previous frame. The spatial characteristics of each block is calculated from the DCT coefficients, and each block is classified as either low-activity or high-activity. For each low-activity block, its DC coefficient value and the DC coefficient values of the surrounding eight neighbor blocks are exploited to predict low frequency AC coefficients which reflect the original coefficients before quantization in the encoding stage. The predicted AC coefficients are inserted into the low activity blocks where blocking artifacts are most noticeable. Subject to available resources, this may be followed by spatial domain post-processing, in which two kinds of low-pass filters are adaptively applied, on a block-by-block basis, according to the classification of the particular block. Strong low-pass filtering is applied in low-activity blocks where the blocking artifacts are most noticeable, whereas weak low-pass filtering is applied in high-activity blocks where ringing noise as well as blocking artifacts may exist. In low activity blocks, the blocking artifacts are reduced by one dimensional horizontal and vertical low-pass filters which are selectively applied in either the horizontal and/or vertical direction depending on the locations and absolute values of the predicted AC coefficients. In high activity blocks, de-blocking and de-ringing is conducted by 2- or 3-tap filters, applied horizontally and/or vertically, which makes the architecture simple.

Growing community of inventors

Foster City, CA, United States of America

William Chen