Koninklijke Philips Corporation N.v.

Philips Electronics North America Corporation

Koninklijke Phillips Electronics N.v.

Mihaela Van Der Schaar

Hayder Radha

Yingwei Chen

Shaomin Peng

Mahesh Balakrishnan

Deepak S Turaga

Sharon Peng

Renatus Josephus Van Der Vleuten

Joseph Patrick Meehan

Coding is provided in which a multi-media object is coded to obtain a bit-stream, and quality information is added to the bit-stream, which quality information indicates a quality of the object in relation to a given position in (or a given part of) the bit-stream. By adding quality information to the bit-stream, jointly storing or transmitting multiple coded objects can be optimized in that the quality of the object can be easily taken into account.

Coding

A method, system and apparatus for improving network transmission efficiency using MDS and Fine Granular Spatial coding is disclosed. In accordance with the method, a video image is partitioned into a plurality of information item frames, each frame is independently layer encoded into a base layer and an enhancement layer, at least one channel of the communication is then selected, each of the base layers are associated with one of the selected communication channels, and each base layer is transmitted over the associated communication channel. In other aspect, a residual bandwidth is determined on each associated channel and a level of an associated enhancement layer comparable to the residual bandwidth is transmitted over the associated channel.

Method and system for improving transmission efficiency using multiple-description layered encoding

A method and system for producing decoding the transmission of high-resolution images transmitted as a low resolution spatially scalable FGS encoded base layer and at least one enhancement layer is presented. The low resolution received base layer is representative of a downscaled image of the original image. In this manner, a minimum resolution base layer is transmitted and higher resolutions may be obtained and utilized depending on the available bandwidth and the receiving system resolution capability. In one aspect of the invention, the base layer is decoded and a quality enhancement is next applied to the base layer. The combined base layer and quality layer video frames are then upscaled and the upscaled image is combined with a decoded spatial enhancement layer information. The spatial enhancement layer information fills in resolution lacking in the upscaled base layer/quality layer image. Thus, a high resolution image is formed. In another aspect of the invention, a temporal layer, containing information regarding image motion, is further applied to the upscaled base layer image to produce a spatially enhanced/temporally enhanced high resolution image.

Method and apparatus for decoding spatially scaled fine granular encoded video signals

Improved efficiency in transmission of FGS encoded video data is achieved by reducing the transmission of overhead information items necessary to decode the received video data. After establishing an initial or first set of criteria for selectively enhancing areas within an image, each subsequent transmission frame includes an indicator that informs a receiving system to decode the current frame using the previously established criteria. The initial criteria include shift factor values specifying selectively enhanced areas of an image; position, displacement vector, size and enhancement factor for at least one area of interest with in an image; and known fixed values contained at both the transmitting and receiving systems.

Method and apparatus for improved efficiency in transmission of fine granular scalable selective enhanced images

A method for decoding a stream of data includes the steps of: identifying a relationship () between quality of a decoded image and a bit rate for decoding the image; selecting a desired quality () of images to be decoded from the stream of data; determining a bit rate () corresponding to the desired quality using the relationship; and decoding a minimum number of bit-planes corresponding to the determined bit rate from the stream of data.

FGS decoder based on quality estimated at the decoder

A method and system for dynamically selectively enhancing desired area or areas of a video image which are FGS encoded. The method comprising the steps of determining at least one of the FGS encoded macroblocks in each of the FGS encoded bit-planes associated with the desired area or portion of the video image, determining an order of transmission of each of the determined FGS encoded macroblocks within the transmission sequence and advancing each of the determined FGS encoded macroblocks in the transmission sequence order corresponding to a known level of enhancement, wherein the advanced FGS encoded macroblocks are contained in a bit-plane having a higher transmission priority. In one aspect of the invention, the desired area may be selected by interactively by a user. In another aspect, the desired area or areas may be selected automatically.

Method and apparatus for dynamic allocation of scalable selective enhanced fine granular encoded images

A video coding technique having motion compensation for bi-directional predicted frames (B-frames) and predicted frames (P-frames). The video coding technique involves a single-loop prediction-based base layer which includes base layer B- and/or P-frames that are generated from 'extended' or 'enhanced' base layer reference frames during base layer coding.

Single-loop motion-compensation fine granular scalability

A method and device is disclosed for encoding a group of video frames. A first number of frames in the group is motion compensated temporally filtered based on multiple frames. A second number of frames in the group is motion compensated temporally filtered based on single reference frames. The first and second number of frames are transformed into wavelet coefficients. Further, the wavelet coefficients are entropy encoded. In one example, the first number of frames are located within a first portion of the group and the second number of frames is located within a second portion of the group. In another example, the first number of frames and the second number of frames are interleaved within the group.

Wavelet based coding using motion compensated filtering based on both single and multiple reference frames

The present invention is directed to a method and device for encoding a group of video frames. According to the present invention, regions in at least one frame in the group are matched to regions in multiple reference frames. The difference between pixel values of the regions in the at least one frame and the regions in the multiple reference frames is calculated. The difference is transformed into wavelet coefficients. The present invention is also directed to a method and device for decoding a group of frames by performing the inverse of the above described encoding.

Motion compensated temporal filtering based on multiple reference frames for wavelet based coding

A method for streaming enhancement layer video frame data on a variable bandwidth network involves coding original uncoded video data with a non-scalable codec to generate I and P base layer frames; generating residual temporal B frames and SNR I and P frames from the original uncoded video data and the base layer frames; coding the temporal and SNR frames with a scalable codec; determining an available bandwidth of the variable bandwidth network; transmitting at least portions of the scalable coded temporal frames when the available bandwidth is a first bandwidth ranging between a minimum bit-rate allocated for transmitting only base layer frames and a predetermined bit-rate which is below a maximum bit-rate of the network; and additionally transmitting remaining portions of the scalable coded temporal frames and at least portions of the scalable coded SNR frames when the available bandwidth is a second bandwidth extending between the predetermined bit-rate and the maximum bit-rate of the network.

Preferred transmission/streaming order of fine-granular scalability

A method and apparatus for coding video includes constructing motion-compensated extended base layer reference frames from base layer frames and at least portions of base layer residual image frames. The motion-compensated extended base layer reference frames are used for bi-directionally or uni-directionally predicting FGS motion-compensated residual image or temporal frames.

Efficiency FGST framework employing higher quality reference frames

A method and apparatus for coding video includes constructing motion-compensated extended reference frames from at least portions of temporal residual image frames. The motion-compensated extended reference frames are used for bi-directionally and/or uni-directionally predicting additional levels of FGS motion-compensated residual image or temporal frames.

A video coding technique having motion compensation within a fine granular scalable coded enhancement layer. In one embodiment, the video coding technique involves a two-loop prediction-based enhancement layer including non-motion-predicted enhancement layer I- and P-frames and motion-predicted enhancement layer B-frames. The motion-predicted enhancement layer B-frames are computed using: 1) motion-prediction from two temporally adjacent differential I- and P- or P- and P-frame residuals, and 2) the differential B-frame residuals obtained by subtracting the decoded base layer B-frame residuals from the original base layer B-frame residuals. In a second embodiment, the enhancement layer further includes motion-predicted enhancement layer P-frames. The motion-predicted enhancement layer P-frames are computed using: 1) motion-prediction from a temporally adjacent differential I- or P-frame residual, and 2) the differential P-frame residual obtained by subtracting the decoded base layer P-frame residual from the original base layer P-frame residual.

Double-loop motion-compensation fine granular scalability

The invention relates to a method and system for modulating an MPEG-4 FGS compressed video stream for variable-bandwidth transmission. A source of a video stream is provided, and a display having means for generating a control input signal for controlling one or more of video compression, error correction code generation, or symbol constellation mapping. The source video stream is compressed using MPEG-4 FGS compression to generate a compressed output video stream. The video compression is optionally performed based upon the control input signal from the display. Error correction code for the compressed video stream may be added, and its generation and addition may be optionally controlled based upon the control input signal from the display. The resulting video symbols may then be mapped to constellations, with this mapping also optionally controlled based upon the control input signal. The resulting compressed output video stream is then transmitted to the display.

Combined MPEG-4 FGS and modulation algorithm for wireless video transmission

A method and apparatus for minimizing prediction drift at low bitrates in a fine granular scalable video coding scheme that utilizes motion compensation in an enhancement layer. The method and apparatus measures motion activity within at least a portion a video; determines whether the measured motion activity is below a predetermined threshold value; codes the portion of the video with the fine granular scalable video coding scheme that utilizes motion compensation in the enhancement layer if the measured motion activity is below the predetermined threshold value; and codes the portion of the video with a fine granular scalable video coding that does not utilize motion compensation in the enhancement layer if the measured motion activity is above the predetermined threshold value.

Growing community of inventors

Ossining, NY, United States of America

Mihaela Van Der Schaar