Sony Corporation

Sony Electronics Inc

Equator Technologies, Inc.

Industrial Technology Research Institute

Pixelworks, Inc.

Jeongnam Youn

Daisuke Hiranaka

Yushin Cho

Alexander Berestov

Takao Yamazaki

Chia-Wen Lin

Ming-Ting Sun

Chris Basoglu

Yuji Wada

Cheung Auyeung

Wen-Hao Wang

In one embodiment, a macroblock is determined in an incoming bitstream. In one example, the macroblock may be a 16&#xd7;16 macroblock for a first coding standard, such as MPEG-2. The macroblock is then divided into a plurality of segmented blocks. For example, an arbitrary sized segment of blocks may be used to divide the macroblock into segmented blocks. In one example, the macroblock is divided into four 8&#xd7;8 segmented blocks. A plurality of statistical measures are then determined for the plurality of segmented blocks. For example, for each segmented block, a sum of absolute differences (SAD) is determined. Then, one or more sizes for the sub-blocks are determined based on the comparison. For example, a variable sub-block size for an AVC bitstream is determined.

Inter sub-mode decision process in a transcoding operation

A camera or camcorder with a multi-view three dimensional (3D) attachment enables acquisition of 3D images and video which are then able to be displayed to a user without the need for specialized glasses. The multi-view 3D attachment captures at least 3 views of the same image from different angles simultaneously on a sensor.

Multi-view video and still 3D capture system

A method and apparatus for preprocessing a video signal prior to its being encoded. During preprocessing, features are extracted from the video signal into feature planes which are each adaptively filtered in response to estimations of feature activity. After feature extraction the signal which remains is referred to as a 'remaining signal', which is then noise filtered to remove a substantial portion of the noise. The feature planes and remaining signal are recombined into a preprocessed output signal configured for receipt and processing by a video encoder. A video encoder receiving the preprocessed video signal can generate higher quality and/or lower bit rate encoded video than one receiving the original source video. Preprocessing according to the invention is particularly well suited for use with encoders adapted to remove temporal and/or spatial redundancy, such as those implemented according to the MPEG2/4, or AVC/H.264 standards.

Adaptive preprocessing method using feature-extracted video maps

Block-based variational image processing provides improved image processing by utilizing portions of an image rather than the entire image. The image is divided into multiple smaller portions, and then iterations to determine a partial differential equation for an image processing application are performed on the smaller portions. After performing the iterations on a portion, the resulting information is able to be stored in an external memory. This results in a much lower bandwidth requirement for the data, enabling the method to be performed in hardware. Additionally, the block-based variational image processing utilizes only a small number of neighboring pixels for each iteration.

Block-based variational image processing method

A display is able to display 3D content in high resolution 2D by utilizing the many views contained in the 3D data and converting the 3D data into 2D data. In some embodiments, the 3D data is converted using shifts in different views of a pixel. In some embodiments, the 3D is converted using shifts in different views of a local pixel and global pixels as well. Displays implementing the 2D high resolution display in addition to a low resolution 3D display are able to display 3D and 2D data depending on a user's preference.

Super-resolution from 3D (3D to 2D conversion) for high quality 2D playback

A compression method using adaptive field data selection is able to compress video or other data while retaining quality. By determining if pixels in a field are able to be predicted, the method is able to retain the information that is not able to be predicted and predict the other information. The method enables significant compression of the data which allows larger data such as high definition videos to be compressed and effectively used.

Compression method using adaptive field data selection

An apparatus and method of performing YUV (or YCrCb) video compression prior to storage within a memory and decompression upon retrieval of the blocks from memory. Compression is performed utilizing a quantizer to compress video data to a desired overall compression ratio R, even though the luma and chroma contributions to compression can differ for each subblock, each preferably selected in response to texture estimation. Selections are made for each subblock to perform either linear or non-linear quantization during compression. Compression is performed without utilizing data from blocks outside of the block being compressed, wherein video blocks can be retrieved and decompressed in any desired order. In one implementation, an encoder non-sequentially selects blocks from memory which are then decompressed and encoded. The compression may be beneficially utilized in a number of different video transmission and storage applications without departing from the invention.

Virtually lossless video data compression

Pixel data is compressed using a combination of variable and fixed quantization for luminance data. The luminance is compared to the result of variable quantization and to the result of the fixed quantization to compute a variable noise and a fixed noise. The result of the variable quantization is selected if the variable noise is less than the fixed noise, otherwise the fixed quantization result is selected if the fixed noise is less than the variable noise. The selected result is transferred to a frame memory. In another aspect of the invention, an intra mode coding decision is forced if the noise corresponding to the selected result exceeds a threshold.

Lossy frame memory compression using intra refresh

A method of deciding a target bit rate for wavelet-based image encoding based on the wavelet coefficients is described. The target bit rate is used for maintaining a high quality image. A minimum target bit rate and a maximum target bit rate are determined. A pseudo code length corresponding to the minimum target bit rate is calculated. The pseudo code length is calculated as a function of the wavelet coefficients. The corresponding pseudo code length is calculated for the maximum target bit rate. A curve between the pseudo code length and target bit rate is derived. Slope of the curve depends on buffer fullness. Using the curve, for a given pseudo code length, an appropriate target bit rate is derived. The derived target bit rate depends on the image content, and computing resources are able to be optimally used to attain similar quality for each image block of an image.

Target bit rate decision method for wavelet-based image compression

A method and apparatus to compress video data to smaller size and embed extra information into the compressed data using the integrated lossy and lossless compressions is described. The method can be used to compress reference frames of a video codec (coder/decoder) combines codec where the small noise is critical and to reduce bus bandwidth. Data transfer between an encoder and an external frame memory connected via an external bus in a video codec is reduced by compressing data from the encoder prior to inputting into the frame memory over the external bus, and decompressing the compressed data from the frame memory after retrieving over the external bus. Reference frames are compressed to variable size without causing any considerable artifact to reduce bus bandwidth between the encoder core and external memory. In the method, lossy and lossless compression is integrated to maximize the compression efficiency.

Video data compression with integrated lossy and lossless compression

A method of detecting existence of visually thin lines in images is described herein. The method includes profile generation, profile analysis and deciding the existence of thin lines. The profile generation includes generating a profile in each direction of the input image. Multiple projection lines are projected along the direction and all of the pixel values along the projection lines are added and averaged to obtain a profile. The profile analysis includes determining presence of thin lines in the input image and also noticing if the direction of the thin lines is similar to that of the projection lines. At the third step, according to the analysis made, existence of thin lines in the image is decided.

Method of detecting the existence of visually sensitive thin lines in a digital image

In one embodiment, a first DCT block of coefficients is received from a transcoding process. The DCT block is of a first size, such as an 8&#xd7;8 size, and in a first coding standard, such as MPEG-2. A distribution of zero and non-zero coefficients is then determined from the first DCT block. For example, areas where non-zero coefficients are found may be determined. This may simplify a corresponding sum of absolute difference (SAD) calculation. The SAD is then calculated for a second DCT block of a second size, such as a 4&#xd7;4 size block, in a second coding standard, such as AVC. The SAD is determined from the coefficients in the first DCT block. Accordingly, the SAD is determined without performing an inverse DCT on the 8&#xd7;8 block and then computing the SAD for a 4&#xd7;4 block. It is then determined if a bypass of a DCT operation for the 4&#xd7;4 block can be performed based on the SAD computed. For example, the SAD is compared to a threshold to determine if a bypass can be performed. In one example, if the coefficients for each 4&#xd7;4 block are determined that they may be quantized to zero, then the DCT operation may be bypassed. Also, quantization, inverse quantization, and inverse DCT steps may also be bypassed.

DCT/Q/IQ/IDCT bypass algorithm in MPEG to AVC/H.264 transcoding

In one embodiment, a method for bypassing transform and quantization steps in a video compression is provided. The method determines one or more threshold values that are computed for a bypass of the transform and quantization steps. A sum of absolute transformed differences (SATD) value from a motion estimation of a signal being compressed using H.26x is received. The SATD value is compared to at least one of the one or more threshold values. It is then determined if the transform and quantization can be bypassed based on the comparison. For example, if the SATD value is less than the at least one of the one or more threshold values, then it may be determined that the coefficients of the transform are expected to be quantized to zero. Accordingly, the transform and quantization steps may be bypassed.

Bypass using sum of absolute transformed differences value (SATD) in a video coding process

An apparatus and method is described for performing texture-decoding within a video decoder processing a series of macroblocks within a video bit-stream. The inventive texture decoding is particularly well suited for implementation within decoders incorporating multiple processors. The method separates data independent operations from data dependent (sequential) operations wherein multiprocessing may be efficiently utilized for processing data independent macroblocks. The method preferably makes an assumption as to prediction direction within the set of data dependent operations. A prediction determination is then made and the assumption corrected if necessary with a transposition within the data independent operations operating on the macroblocks. The method reduces the computational overhead and provides for efficiently segmenting operations within a multiprocessing decoder.

Method of real time MPEG-4 texture decoding for a multiprocessor environment

A method of rapidly generating motion vector predictions based on vertical and horizontal position categorization of macroblocks within a video object plane (VOP) for use within a video decoder or encoder. By way of example, the location of the subject macroblock is categorized vertically as either Upper_Edge or Not_Upper_Edge, and horizontally as either Left_Edge, Right_Edge, or Not_Edge. The position categories are utilized in conjunction with selected block number (Blockthrough Block) within the luminance macroblock within a decision-tree which generates three predictors MVMVand MVThe prediction rules may be implemented within hardware, software, or combinations thereof, within both video encoders and decoders, such as according to the MPEG-4, H.263, or similar standards.

Growing community of inventors

San Jose, CA, United States of America

Jeongnam Youn