Dolby Laboratories Licensing Corporation

Qing Song

Guan-Ming Su

Harshad Kadu

Qian Chen

Neeraj J Gadgil

Arun Raj

Peng Yin

Tao Chen

Qiang Zhu

Yoon Yung Lee

Sankaranarayanan Piramanayagam

A set of tensor-product B-Spline (TPB) basis functions is determined. A set of selected TPB prediction parameters to be used with the set of TPB basis functions for generating predicted image data in mapped images from source image data in source images of a source color grade is generated. The set of selected TPB prediction parameters is generated by minimizing differences between the predicted image data in the mapped images and reference image data in reference images of a reference color grade. The reference images correspond to the source images and depict same visual content as depicted by the source images. The set of selected TPB prediction parameters is encoded in a video signal as a part of image metadata along with the source image data in the source images. The mapped images are caused to be reconstructed and rendered with a recipient device of the video signal.

Tensor-product b-spline predictor

A device includes an electronic processor configured to define a first set of sample pixels from a set of sample pixels determined from received video data according to a first electro-optical transfer function (EOTF) in a first color representation of a first color space; convert the first set of sample pixels to a second EOTF via a mapping function, producing a second set of sample pixels according to the second EOTF; convert the first and second set of sample pixels from the first color representation to a second color representation of the first color space; determine a backward reshaping function by repeatedly applying and adjusting a sample backward reshaping function so as to minimize a difference between predicted pixel values obtained by applying the sample backward reshaping function to the pixels of the converted first set and the pixels of the converted second set.

Electro-optical transfer function conversion and signal legalization

A backward reshaping mapping table is initially generated as an inverse of a forward reshaping mapping table. The backward reshaping mapping table is updated by replacing the content-mapped luminance codewords with forward reshaped luminance codewords generated by applying a luminance forward mapping to the sampled luminance codewords. The luminance forward mapping is constructed from the forward reshaping mapping table. The backward reshaping mapping table and the luminance forward mapping are used to generate backward reshaping mappings for creating a reconstructed image from a forward reshaped image. The forward reshaped image is encoded, in a video signal, along with image metadata specifying the backward reshaping mappings. A recipient device of the video signal applies the backward reshaping mappings to the forward reshaped image to create the reconstructed image of the second dynamic range.

Adjustable trade-off between quality and computation complexity in video codecs

Methods and systems for reducing banding artifacts when displaying images are described. Identified image bands are filtered using an adaptive sparse finite response filter, where the tap-distance in the sparse filter is adapted according to an estimated width of each image band. Image debanding may be performed across multiple pixel orientations, such as rows, columns, a 45-degree angle, or a &#x2212;45-degree angle. Given a threshold to decide whether sparse filtering needs to be performed or not, an iterative debanding process is also proposed.

Image debanding using adaptive sparse filtering

Methods and systems for generating an interpolated reshaping function for the efficient coding of high-dynamic range images are provided. The interpolated reshaping function is constructed based on a set of pre-computed basis reshaping functions. Interpolation schemes are derived for pre-computed basis reshaping functions represented as look-up tables, multi-segment polynomials, or matrices of coefficients in a multivariate, multi-regression representation. Encoders and decoders using asymmetric reshaping and interpolated reshaping functions for mobile applications are also presented.

Interpolation of reshaping functions

Methods and systems for mapping images from a first dynamic range to a second dynamic range using a set of reference color-graded images and neural networks are described. Given a first and a second image representing the same scene but at a different dynamic range, a neural network (NN) model is selected from a variety of NN models to determine an output image which approximates the second image based on the first image and the second image. The parameters of the selected NN model are derived according to an optimizing criterion, the first image and the second image, wherein the parameters include node weights and/or node biases for nodes in the layers of the selected NN model. Example HDR to SDR mappings using global-mapping and local-mapping representations are provided.

HDR image representations using neural network mappings

Methods and systems for reducing banding artifacts when displaying high-dynamic-range images are described. Given an input image in a first dynamic range, and an input backward reshaping function mapping codewords from the first dynamic range to a second dynamic range, wherein the second dynamic range is equal or higher than the first dynamic range, statistical data based on the input image and the input backward reshaping function are generated to estimate the risk of banding artifacts in a target image in the second dynamic range generated by applying the input backward reshaping function to the input image. Separate banding alleviation algorithms are applied in the darks and highlights parts of the first dynamic range to generate a modified backward reshaping function, which when applied to the input image to generate the target image eliminates or reduces banding in the target image.

Reducing banding artifacts in HDR imaging via adaptive SDR-to-HDR reshaping functions

3D mapping statistics are generated for a first image of a first dynamic range and a second image of a second dynamic range different from the first dynamic range. Multivariate multiple regression (MMR) coefficients are generated by solving an optimization problem formulated using an MMR matrix built with the 3D mapping statistics without a letterbox constraint, and used to generate chroma mappings for predicting chroma codeword values of the second image. It is determined whether a letterbox exists in the images. If so, it is determined whether the chroma mappings accurately predict chroma codeword values in the second image. A reconstructed image generated by a recipient device by backward reshaping one of the images is rendered by a display device operating in conjunction with the recipient device.

High-fidelity full reference and high-efficiency reduced reference encoding in end-to-end single-layer backward compatible encoding pipeline

Methods and systems for reducing banding artifacts when displaying high-dynamic-range images reconstructed from coded reshaped images are described. Given an input image in a high dynamic range (HDR) which is mapped to a second image in a second dynamic range, banding artifacts in a reconstructed HDR image generated using the second image are reduced by a) in darks and mid-tone regions of the input image, adding noise to the input image before being mapped to the second image, and b) in highlights regions of the input image, modifying an input backward reshaping function, wherein the modified backward reshaping function will be used by a decoder to map a decoded version of the second image to the reconstructed HDR image. An example noise generation technique using simulated film-grain noise is provided.

Reducing banding artifacts in backward-compatible HDR imaging

Real-time forward reshaping, comprising selecting a statistical sliding window that indexes with the current frame, having also, a look-back frame and a look-ahead frame, determining whether they are part of the current scene, determining a noise parameter, a luma transfer function and a luma forward reshaping function based on the luma transfer function and the noise parameter within the current scene, selecting a central tendency sliding window of the current frame and the look-back frame within the current scene, and determining a central tendency luma forward reshaping function. The chroma reshaping comprises analyzing statistics for the extended dynamic range (EDR) weights and EDR upper bounds, mapping these to standard dynamic range (SDR) weights and SDR upper bounds based on the central tendency luma forward reshaping function, determining a chroma content-dependent polynomial and a central tendency chroma forward reshaping polynomial and generating chroma MMR coefficients.

Real-time reshaping of single-layer backwards-compatible codec

Methods and systems for adaptive chroma reshaping are discussed. Given an input image, a luma-reshaped image is first generated based on its luma component. For each chroma component of the input image, the range of the pixel values in the luma reshaped image is divided into bins, and for each bin a maximal scale factor is generated based on the chroma pixel values in the input image corresponding to the pixels of the luma reshaped image in the bin. A forward reshaping function is generated based on a reference reshaped function and the maximal scale factors, and reshaped chroma pixel values for the chroma component are generated based on the forward reshaping function and the corresponding pixel values in the luma reshaped image. Implementations options using look-up tables for mobile platforms with limited computational resources are also described.

Chroma reshaping for high dynamic range images

A standard dynamic range (SDR) image is received. Composer metadata of the first level through the N-th level is generated. Composer metadata of the j-th level is generated based on the composer metadata of the first level through (j&#x2212;1)-th level. The composer metadata of the first level through the composer metadata of the j-th level is to be used for mapping the SDR image to the j-th target image specifically optimized for the j-th reference target display. The SDR image is encoded with the composer metadata of the first level through the k-th level in an output SDR video signal, where 1&#x3c;=k&#x3c;=N. A display device renders a display image derived from a composed target image composed from the SDR image based on the composer metadata of the first level through the k-th level in the output SDR video signal.

Single-layer progressive coding for supporting multi-capability HDR composition

Methods for screen-adaptive decoding of video with high dynamic range (HDR) are described. The methods combine the traditional compositing and display management steps into one screen-adaptive compositing step. Given decoded standard dynamic range (SDR) input data, metadata related to the prediction of output HDR data in a reference dynamic range, and the dynamic range of a target display, new output luma and chroma prediction functions are generated that map directly the input SDR data to output HDR data in the target dynamic range, thus eliminating the display management step.

Screen-adaptive decoding of high dynamic range video

A sparse FIR filter can be used to process an image in order to rectify imaging artifacts. In a first example application, the sparse FIR filter is applied as a selective sparse FIR filter that examines a set of selected neighboring pixels of an original pixel in order to identify smooth areas of the image and to selectively apply filtering to only the smooth areas of the image. The parameters of selective filtering are selected based on the characteristics of an inter-layer predictor. In a second example application, the sparse FIR filter is applied as an edge aware selective sparse FIR filter that examines additional neighboring pixels to the set of selected pixels in order to identify edges and carry out selective filtering of smooth areas of the image. Examples for detecting and removing banding artifacts during the coding of high-dynamic range images are provided.

Growing community of inventors

Sunnyvale, CA, United States of America

Qing Song