Hewlett-packard Development Company, L.p.

Leland Stanford Junior University

Itschak Weissman

Gadiel Seroussi

Erik Ordentlich

Marcelo Weinberger

Sergio Verdu

Giovanni Motta

Krishnamurthy Viswanathan

Sergio Verdú

Shirin Jalali

The present invention provides methods for universal lossy compression that provide performance at or near the rate-distortion limit and that are based on universal, implementable lossy source coding algorithms.

Universal lossy compression methods

Various embodiments of the present invention relate to a discrete denoiser that replaces symbols in a received, noisy signal with replacement symbols in order to produce a recovered signal less distorted with respect to an originally transmitted, clean signal than the received, noisy signal. Certain, initially developed discrete denoisers employ an analysis of the number of occurrences of metasymbols within the received, noisy signal in order to select symbols for replacement, and to select the replacement symbols for the symbols that are replaced. Denoisers that represent examples of the present invention use blended counts that are combinations of the occurrences of metasymbol families within a noisy signal to determine the symbols to be replaced and the replacement symbols corresponding to them.

Discrete denoising using blended counts

In various embodiments of the present invention, optimal or near-optimal multidirectional context sets for a particular data-and/or-signal analysis or processing task are determined by selecting a maximum context size, generating a set of leaf nodes corresponding to those maximally sized contexts that occur in the data or signal to be processed or analyzed, and then building up and concurrently pruning, level by level, a multidirectional optimal context tree constructing one of potentially many optimal or near-optimal context trees in which leaf nodes represent the context of a near-optimal or optimal context set that may contain contexts of different sizes and geometries. Pruning is carried out using a problem-domain-related weighting function applicable to nodes and subtrees within the context tree. In one described embodiment, a bi-directional context tree suitable for a signal denoising application is constructed using, as the weighting function, an estimated loss function.

Method and system for determining an optimal or near optimal set of contexts by constructing a multi-directional context tree

A method and apparatus for processing a received digital signal that has been corrupted by a channel is disclosed. The method includes storing the received digital signal and receiving a partially corrected sequence of symbols that includes an output of a preliminary denoising system operating on the received digital signal. Information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by a symbol having the value J is utilized to generate a processed digital signal by replacing each symbol having a value I in a context of that symbol in the received digital signal with a symbol having a value J if replacement reduces a measure of overall signal degradation in the processed digital signal relative to the received digital signal as measured by the degradation function and the partially corrected sequence of symbols.

Enhanced denoising system

Various embodiments of the present invention provide methods and systems for determining, representing, and using variable-length contexts in a variety of different computational applications. In one embodiment of the present invention, a balanced tree is used to represent all possible contexts of a fixed length, where the depth of the balanced tree is equal to the fixed length of the considered contexts. Then, in the embodiment, a pruning technique is used to sequentially coalesce the children of particular nodes in the tree in order to produce an unbalanced tree representing a set of variable-length contexts. The pruning method is selected, in one embodiment, to coalesce nodes, and, by doing so, to truncate the tree according to statistical considerations in order to produce a representation of a variably sized context model suitable for a particular application.

Method and system for producing variable length context models

Denoising such as discrete universal denoising (DUDE) that scans a noisy signal in an attempt to characterize probabilities of finding symbol values in a particular context in a clean signal can perform a rough denoising on the noisy signal and identify contexts from a roughly denoised signal. The rough denoising improves estimation of the statistical properties of the clean signal by reducing the false differentiation of contexts that noise can otherwise create. Statistical information regarding occurrences of symbols in the noisy signal and corresponding contexts in the roughly denoised signal can then be used to denoise the noisy signal. The specifics of the rough denoising can be chosen based on knowledge of the noise or of the clean data. Alternatively, the DUDE can be used in an iterative fashion where the denoised signal produced from a prior iteration provides the contexts for the next iteration.

Context identification using a denoised signal

Various embodiments of the present invention provide a compression method and system that compresses received data by first denoising the data and then losslessly compressing the denoised data. Denoising removes high entropy features of the data to produce lower entropy, denoised data that can be efficiently compressed by a lossless compression technique. One embodiment of the invention is a universal lossy compression method obtained by cascading a denoising technique with a universal lossless compression method. Alternative embodiments include methods obtained by cascading a denoising technique with one or more lossy or lossless compression methods.

Methods for compression using a denoiser

In various embodiments of the present invention, a noisy signal denoiser is tuned and optimized by selecting denoiser parameters that provide relatively highly compressible denoiser output. When the original signal can be compared to the output of a denoiser, the denoiser can be accurately tuned and adjusted in order to produce a denoised signal that resembles as closely as possible the clear signal originally transmitted through a noise-introducing channel. However, when the clear signal is not available, as in many communications applications, other methods are needed. By adjusting the parameters to provide a denoised signal that is globally or locally maximally compressible, the denoiser can be optimized despite inaccessibility of the original, clear signal.

Method and system for optimizing denoising parameters using compressibility

An apparatus for operating on a received signal that includes a noise-free signal that has been corrupted by a channel is disclosed. A memory stores a channel corruption function specifying the probability that a symbol having a value I was converted to a symbol having a value J by the channel, and a degradation function measuring the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller parses one of the received signal or the processed signal into phrases, and replaces one of the symbol having a value I in a context of that symbol in the received signal with a symbol having a value J if the replacement would reduce the estimated overall signal degradation in the processed signal. The context of a symbol depends on the phrase associated with the symbol.

Enhanced denoising system utilizing incremental parsing

A denoising process statistically processes a series of frames of a motion picture to construct respective data structures for the frames. Each data structure indicates for each of multiple contexts, occurrences of symbols that have the same context and are in the corresponding one of the frames. The data structures for multiple frames are combined to construct an enhanced data structure for one of the frames, and symbols in that frame are replaced with values determined using the enhanced data structure.

Denoising video

A discrete, universal denoising method is applied to a noisy signal for which the source alphabet is typically large. The method exploits a priori information regarding expected characteristics of the signal. In particular, using characteristics of a continuous tone image such as continuity and small-scale symmetry allows definition of context classes containing large numbers of image contexts having similar statistical characteristics. Use of the context classes allows extraction of more reliable indications of the characteristic of a clean signal.

Context-based denoiser that simultaneously updates probabilities for multiple contexts

A method of and system for generating reliability information for a noisy signal received through a noise-introducing channel. In one embodiment, symbol-transition probabilities are determined for the noise-introducing channel. Occurrences of metasymbols in the noisy signal are counted, each metasymbol providing a context for a symbol of the metasymbol. For each metasymbol occurring in the noisy signal, reliability information for each possible value of the symbol of the metasymbol is determined, the reliability information representing a probability that the value in the original signal corresponding to the symbol of the metasymbol assumed each of the possible values. In another embodiment, error correction coding may be performed by adding redundant data to an original signal prior to transmission by the noise-introducing channel and performing error correction decoding after transmission.

Discrete universal denoising with reliability information

An apparatus and method for processing a received signal that has been corrupted by a channel to generate a processed signal having less signal corruption than the received signal is disclosed. The apparatus stores the received signal, information specifying the probability that a symbol having a value I will be converted to a symbol having a value J by the channel, and information specifying a signal degradation function that measures the signal degradation that occurs if a symbol having the value I is replaced by symbol having a value J. The controller replaces each symbol having a value I in a context of that symbol in the received signal with a symbol having a value J that minimizes the overall signal degradation in the processed signal relative to the underlying noise-free signal as estimated via the observed statistics within that context.

Growing community of inventors

Menlo Park, CA, United States of America

Itschak Weissman