Sharp Laboratories of America, Inc

Shaw-min Lei

Jin Li

Jordi Ribas-Corbera

Wenjun Zeng

Christopher A Segall

Larry Alan Westerman

Kristine Elizabeth Matthews

Gary Alan Feather

Eugene Paul Martinez, Jr

Prasanna L Modem

Jordi Ribas

Adaptive up-sample filtering is used to improve compression efficiency of spatially scalable coding systems by more effectively predicting the high-resolution (enhanced-layer) video (or image) from the low-resolution lower-layer video (or image). Different up-sample filters adaptive to local image properties are selectively used for different portions of a low resolution frame to generate a better up-sampled image. Selection between different up-sample filters is determined by a variety of different information available to both the encoder and decoder. In one embodiment, the up-sample filters are selected by the encoder and then explicitly identified to the decoder. Other techniques are then used to minimize the cost of transmitting the up-sample filter identifiers. In alternative embodiments, the encoder and decoder independently make up-sample filters selections.

Method and apparatus for adaptive up-scaling for spatially scalable coding

A method for processing saturated video intervals in a video sequence. The method either removes or retains the saturated interval. If the interval is removed, the sequence is altered to accommodate the removal. The accommodation is either complete elimination, substitution of a derived interval, or substitution by repeating adjacent intervals. If the interval is retained, information is extracted while limiting the effects of the saturation. The interval can either be encoded with lower data rates, with increased data rates being applied to subsequent intervals, the temporal prediction encoding can be forced to a B interval, an I interval can be inserted, or the sequence restarted, or a coupled encoder decoder system can be set up that uses the same transform and clipping steps to limit the effects of saturation.

Method for processing saturated intervals in video sequences

A rate-distortion optimized embedding (RDE) coder optimizes rate-distortion performance by coding information bits in the order of their R-D slope. RDE allocates the available coding bits first to the information bit with the steepest R-D slope, which indicates the largest distortion decrease per coding bit. The resultant RDE bitstream can be truncated at any point and still maintain an optimal R-D performance. To avoid the overhead of coding order transmission, an expected R-D slope is calculated by both the encoder and the decoder from previous RDE coded bits. A probability estimation table from a QM arithmetic coder allows the R-D slope to be derived using a lookup table operation. The rate-distortion optimised embedding (RDE) coder significantly improves the coding efficiency over a wide range of bit rates.

Embedded image coder with rate-distortion optimization

Block and frame skipping decides which image regions, blocks or frames in a video frame or series of frames should be encoded. Block skipping detects in advance which of the blocks in a video frame will produce little or no bits, so that the encoder can skip the encoding process for these blocks. Block skipping uses an iterative procedure to determine the optimal energy threshold value for each image frame. All the image blocks whose pixel energy is below the optimal threshold for the frame are skipped. Frame skipping predicts a distortion or image quality value for each frame. If some frames have a predicted image quality smaller than a preset value, these frames are allocated more bits and if, as a result, too few bits are left for future frames, the latter frames are skipped. This selective frame skipping guarantees that the quality of the non-skipped frames is above the preset value.

Method and apparatus for selecting image data to skip when encoding digital video

An adaptive image coding method and system are disclosed. The system accepts an input image, divides it into image segments, and assigns each segment to a wavelet transform filter from a bank of such filters for transformation. The bank preferably comprises filters adapted for different types of image content, e.g., sharp edges, slowly-varying contours, etc. Filters are preferably assigned so as to produce minimal distortion for their assigned image segments at a given target bit rate, each filter produces transform coefficients for its segment using scale and subband settings common to the entire image. The valid coefficients for each segment are then combined in a composite wavelet coefficient image, which resembles a single wavelet transform of an entire image&mdash;although different filters are used to create different portions of the coefficient image. The composite image allows joint, rate-distortion optimized coding for a segmented image. Joint coding allocates bits between the transforms of the image segments optimally, and produces an easily scaleable bitstream.

Joint coding method for images and videos with multiple arbitrarily shaped segments or objects

Re-indexing for efficient compression of palettized images

Digital image scrambling for image coding systems

An embedded coding scheme is used to dynamically changes the size of compressed images and as a result image quality according to the number of stored pictures. As more images are captured, the image storage device makes room for newly captured images by truncating existing compressed embedded coded bitstreams of previously stored images. The embedded digital image storage device can thus store a virtually unlimited number of images by dynamically trading-off between the number of stored images and image quality.

Dynamic management of embedded coded images in a digital storage device

Block and frame skipping decides which image regions, blocks or frames in a video frame or series of frames should be encoded. Block skipping detects in advance which of the blocks in a video frame will produce little or no bits, so that the encoder can skip the encoding process for these blocks. Block skipping uses an iterative procedure to determine the optimal energy threshold value for each image frame. All the image blocks whose pixel energy is below the optimal threshold for the frame are skipped. Frame skipping predicts a distortion or image quality value for each frame. If some frames have a predicted image quality smaller than a preset value, these frames are allocated more bits and if, as a result, too few bits are left for fixture frames, the latter frames are skipped. This selective frame skipping guarantees that the quality of the non-skipped frames is above the preset value.

A digital video encoder and method for its operation are disclosed. This encoder comprises a frame-layer rate controller that bases a rate-distortion-optimized target bit assignment T on an energy estimate for frame j, an average energy estimate for a group of frames, and the desired overall bit rate. If the encoder uses motion compensation, motion bits used for frame j and an average motion bits estimate for a group of frames may also be used by the frame-layer rate controller. Preferably, the desired delay in the system affects how average estimates are computed by the encoder. The frame-layer rate controller of the video encoder above may also utilize buffer protection logic during target bit assignment. This logic adjusts an initial target bit assignment T based on the transmission bit rate and desired delay. A primary benefit of the frame-layer rate controller disclosed herein is its basis in a rate-distortion optimal solution to the frame-layer rate control problem. Additional benefits are provided by the explicit consideration of buffer delay during selection of a target bit assignment: rate control robustness allows one controller to be used in a range of delay situations (or even a varying delay situation); and buffer underflow/overflow can be largely eliminated, even in very low delay implementations.

Method and apparatus for a delay-adaptive rate control scheme for the frame layer

A quad-tree embedded image coding technique is used in combination with a bit-plane encoding technique to provide an efficient and low complexity embedded image coding system. A simple quad-tree method identifies coefficients as significant, insignificant, or refinement at each successive quantization level. The quad-tree technique is used instead of the zero-tree or hierarchical tree used in previous encoders.

Quad-tree embedded image compression and decompression method and apparatus

Methods and apparatus for compressing and decompressing reference frames in a video decoder present a low-complexity approach to conserving reference frame memory. At the same time, this approach provides PSNR and drift-reduction performance comparable to competing methods of relatively high-complexity, allowing typical compression rates of 2:1 to 8:1 to be used with good performance. In a preferred embodiment, reference frames are blocked, and each block is Haar wavelet transformed. The subbands of the wavelet transform with similar expected bit-level run-length statistics are grouped together, and each group is embedded coded using Golomb-Rice codes with a parameter look-up adaption method. The embedded code is stored in a compressed frame memory, indexed by block, for later retrieval and decompression when the reference block is needed by the video decoder.

Compression and decompression of reference frames in a video decoder

An arbitrary shape wavelet transform with phase alignment (ASWP) is used to transform an arbitrary shaped object in an image. The phase of an odd tap wavelet filter is aligned so that a low pass filter is always centered at an even index, and a high pass filter is always centered at an odd index. The phase of an even tap wavelet filter is aligned so that the low pass filter and the high pass filter are both centered at index 2i&plus;0.5, i.e., a half index past the even index. The objects for odd tap wavelet filters are each separately symmetrically extended by mirroring the objects from the opposite ends but not mirroring the end pixels. The objects for the even tap filter is symmetrically extended by mirroring the pixels from the opposite ends of the objects including mirroring the end pixels. The phase adjusted-symmetrically extended objects are then transformed.

Growing community of inventors

Camas, WA, United States of America

Shaw-min Lei