Slipstream Data Inc.

Research in Motion Corporation

Blackberry Corporation

Synamedia Limited

En-Hui Yang

Longji Wang

En-hui Yang

Ajit Singh

Cheng-Yu Pai

Guixing Wu

Brian Lamb

Stephen Warrington

Techniques for machine learning based coded size estimation in rate control of video encoding are described. An encoder in accordance with various embodiments uses one or more machine learning approaches to facilitate the rate control of video encoding. When training one or more neural network models, the relationship between the coded size and unit characteristics (e.g., picture pixels and/or picture features) is learned from past encoding. The encoder then uses the trained model(s) to estimate the coded size through model inference with improved accuracy. In some embodiments, the trained model(s) are integrated into the encoder for direct model inference. The direct model inference reduces the overhead of referencing application programming interfaces (APIs) provided by a separate machine learning platform, thus making the rate control methods and systems described herein useful in real time video encoding.

Machine learning based coded size estimation in rate control of video encoding

A method, system and computer software product for improving rate-distortion performance while remaining faithful to JPEG/MPEG syntax, involving joint optimization of Huffman tables, quantization step sizes and quantized coefficients of a JPEG/MPEG encoder. This involves finding the optimal coefficient indices in the form of (run, size) pairs. By employing an interative process including this search for optimal coefficient indices, joint improvement of run-length coding, Huffman coding and quantization table selection may be achieved. Additionally, the compression of quantized DC coefficients may also be improved using a trellis-structure.

Method, system and computer program product for optimization of data compression with iterative cost function

The present disclosure relates to methods and systems for progressively delivering multimedia content over a network. According to one aspect, a system for delivering multimedia content through a network includes a server side sub-system and a client side sub-system. The server side sub-system is configured to: concurrently send a first predetermined portion of each of the plurality of objects through the network, wherein the first predetermined portion is less than a complete object for at least one of the plurality of objects; and after sending the first predetermined portion of each of the plurality of objects, successively send subsequent predetermined portions of each of the plurality of objects until all portions of the plurality of objects have been sent. The client side sub-system is configured to: receive the first predetermined portion of each of the plurality of objects through the network; and after receiving the first predetermined portion of each of the plurality of objects, successively receive subsequent predetermined portions of each of the plurality of objects until all portions of each of the plurality of objects have been received.

System and method for progressive delivery of multimedia objects

A method, system and computer program product for progressively encoding a digitized color image is provided. This involves: initializing a tree structure with at least one starting leaf node; determining at least one representative color for each starting leaf node; and growing the tree structure and updating an encoded image by repeatedly (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost; (ii) creating the two new leaf nodes by re-allocating each color in n; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the selected leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.

Method, system and computer program product for providing entropy constrained color splitting for palette images with color-wise splitting

A sequence of n coefficients is compressed by determining a cost-determined sequence of n coefficient indices represented by a cost-determined sequence of (run, index derivative) pairs under a given quantization table and run-index derivative coding distribution, wherein each sequence of (run, index derivative) pairs defines a corresponding sequence of coefficient indices such that (i) each index in the corresponding sequence of coefficient indices is a digital number, (ii) the corresponding sequence of coefficient indices includes a plurality of values including a special value, and (iii) each (run, index derivative) pair defines a run value representing a number of consecutive indices of the special value, and an index-based value derived from a value of the index following the number of consecutive indices of the special value.

Method, system and computer program product for optimization of data compression

A method, system and computer program product are provided for progressively encoding a digitized color image using a data processing system, the digitized color image being provided by assigning each of the M distinct colors to at least one pixel in the set of pixels. This involves initializing and growing the tree structure by selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is based on 1) a determined associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and 2) a determined associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes.

Method, system and computer program product for entropy constrained color splitting for palette images with pixel-wise splitting

To resize a compressed image, first and second scaling parameters P and Q are determined in accordance with the block numbers L and M and the scaling ratio L/M or M/L. A non-uniform, or uneven, sampling in the DCT domain is then applied to coefficients of successive blocks in the compressed image in accordance with the scaling parameters. In an embodiment, P blocks are sampled and transformed from a given block length to |P| according to a |P|-point IDCT, while Q blocks are sampled and transformed from a given block length to |Q| according to a |Q|-point IDCT. The non-uniformly sampled and transformed pixel domain samples are then regrouped into a predetermined block size and transformed back to generate the DCT coefficient of the compressed image. The proposed method significantly reduces the computational complexity compared with other DCT domain arbitrary ratio image resizing approach.

Arbitrary ratio image resizing in the DCT domain

A method for optimization of rate-distortion for Advanced Audio Coding (AAC). The method provides for the identification of quantized spectral coefficient sequences for optimization of rate-distortion. The method also provides joint optimization of scale factors, Huffman codebooks and quantized spectral coefficient sequences for minimization of a rate-distortion cost. The method provides an iterative rate-distortion optimization algorithm for AAC encoding. In each iteration, the method first finds the optimal scale factors and quantized spectral coefficients when Huffman codebooks are fixed, then updates Huffman codebooks and quantized spectral coefficients given the optimized scale factors. The iterations may be applied until a predetermined threshold is attained.

Rate-distortion optimization for advanced audio coding

A sequence of n coefficients is compressed by determining a cost-determined sequence of n coefficient indices represented by a cost-determined sequence of (run, index derivative) pairs under a given quantization table and run-index derivative coding distribution, wherein each sequence of (run, index derivative) pairs defines a corresponding sequence of coefficient indices such that (i) each index in the corresponding sequence of coefficient indices is a digital number, (ii) the corresponding sequence of coefficient indices includes a plurality of values including a special value, and (iii) each (run, index derivative) pair defines a run value representing a number of consecutive indices of the special value, and an index-based value derived from a value of the index following the number of consecutive indices of the special value. This involves (a) using the given quantization table and run-index derivative coding distribution to formulate a cost function for a plurality of possible sequences of (run, index derivative) pairs; (b) applying the cost function to each possible sequence in the plurality of possible sequences of (run, index derivative) pairs to determine an associated cost; and, (c) selecting the cost-determined sequence of (run, index derivative) pairs from the plurality of possible sequences of (run, index derivative) pairs based on the associated cost of each of the plurality of possible sequences of (run, index derivative) pairs; and encoding the corresponding selected cost-determined sequence of (run, index derivative) pairs using entropy coding based on a run-index derivative coding distribution.

A method, system and computer program product for progressively encoding a digitized color image is provided. This involves: initializing a tree structure with at least one starting leaf node; determining at least one representative color for each starting leaf node; and growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost; (ii) creating the two new leaf nodes by re-allocating each color in n; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.

A method and system for arbitrary resizing of a compressed image in the discrete cosine transform (DCT) domain. First and second scaling parameters P and Q are determined in accordance with the block numbers L and M and the scaling ratio L/M or M/L. A non-uniform, or uneven, sampling in the DCT domain is then applied to coefficients of successive blocks in the compressed image in accordance with the scaling parameters. In an embodiment, P blocks are sampled and transformed from a given block length to |P| according to a |P|-point IDCT, while Q blocks are sampled and transformed from a given block length to |Q| according to a |Q|-point IDCT. The non-uniformly sampled and transformed pixel domain samples are then regrouped into a predetermined block size and transformed back to generate the DCT coefficient of the compressed image. The proposed method significantly reduces the computational complexity compared with other DCT domain arbitrary ratio image resizing approach. It also facilitates the practical implementations since only the fast implementations of IDCT with length N (1&#x3c;N&#x3c;9) and DCT with length 8 are required.

Growing community of inventors

Waterloo, Canada

Longji Wang