Dolby Laboratories Licensing Corporation

Dolby Laboratoreis Licensing Corporation

Harshad Kadu

Peng Yin

Tao Chen

Guan-Ming Su

Qing Song

Neeraj J Gadgil

Qian Chen

Tsung-Wei Huang

Samir N Hulyalkar

Walter J Husak

Sheng Qu

Jon Scott Miller

Yoon Yung Lee

Per Jonas Andreas Klittmark

Bihan Wen

Hariharan Ganapathy Kathirvelu

Yee Jin Lee

Subhayan Mukherjee

Janos Horvath

Bharath Vishwanath

Ji Qi

Hanyang Sun

Tensor-Product B-splines (TPB) have been shown to improve video quality when used to represent reshaping functions to map reshaped standard dynamic range content into high dynamic range (HDR) content; however, TPB prediction is computationally intensive and may not be supported by legacy devices. Methods and systems for backwards-compatible signaling of TPB-related metadata and a fast TPB prediction method are presented to overcome both of these limitations. Computation overhead for a TPB-based 3D look-up table is reduced by using temporary two-dimensional arrays. A remapping of the most significant bits of a legacy bit-depth parameter allows for backwards compatibility.

Tensor-product B-spline prediction for HDR video in mobile applications

Methods, systems, and bitstream syntax are described for metadata signaling and film-grain parameter adaptation based on a viewing environment which may differ from a reference environment. Example adaptation models are provided for viewing parameters that include: ambient room illumination, viewing distance, and pixels per inch in a target display. Example systems include a single reference viewing environment model and a multi-reference viewing environment model supporting adaptation of film-grain model parameters via adaptation functions or interpolation.

Film grain parameters adaptation based on viewing environment

Given input HDR and SDR images representing the same scene, a prediction model to predict the HDR image from a compressed representation of the input SDR image is generated as follows: a) generate noise data based at least on the characteristics of the HDR image b) generate a noisy SDR image by adding the noise data to the SDR image c) generate an augmented HDR data set and an augmented SDR data set by using the input HDR and SDR images and the noisy SDR image d) generate a prediction model to predict the augmented HDR data set based on the augmented SDR data set and e) solve the prediction model according to a minimization-error criterion to generate a set of prediction parameters to be transmitted to a decoder together with a compressed representation of the input SDR image to reconstruct an approximation of the input HDR image.

Image prediction for HDR imaging in open-loop codecs

In an encoder, a high-dynamic range (HDR) image is encoded using a family of local forward reshaping functions selected according to an array of forward mapping indices (FMI) indicating which local forward reshaping function needs to be used for each pixel in the HDR image to generate a reshaped standard dynamic range (SDR) image. A decoder, given the reshaped SDR image, iteratively generates a reconstructed HDR image and estimated reshaped SDR images by adjusting a local FMI array and a local array of backward mapping indices (BMI) until an error metric related to the difference between the local BMI and FMI arrays and the difference between the estimate SDR images and the reshaped SDR image satisfy a convergence criterion. Techniques for generating families of local forward reshaping functions and local backward reshaping functions based on a global forward reshaping function are also presented.

Blind local reshaping in HDR imaging

A forward reshaping mapping is generated to map a source image to a corresponding forward reshaped image of a lower dynamic range. The source image is spatially downsampled to generate a resized image into which noise is injected to generate a noise injected image. The forward reshaping mapping is applied to map the noise injected image to generate a noise embedded image of the lower dynamic range. A video signal is encoded with the noise embedded image and delivered to a recipient device for the recipient device to render a display image generated from the noise embedded image.

Adaptive streaming with false contouring alleviation

Backward reshaping metadata prediction models are trained with training SDR images and corresponding training HDR images. Content creation user input to define user adjusted HDR appearances for the corresponding training HDR images is received. Content-creation-user-specific modified backward reshaping metadata prediction models are generated based on the trained prediction models and the content creation user input. The content-creation-user-specific modified prediction models are used to predict operational parameter values of content-creation-user-specific backward reshaping mappings for backward reshaping SDR images into mapped HDR images of at least one content-creation-user-adjusted HDR appearance.

Efficient user-defined SDR-to-HDR conversion with model templates

An input image of a first bit depth in an input domain is received. Forward reshaping operations are performed on the input image to generate a forward reshaped image of a second bit depth in a reshaping domain. An image container containing image data derived from the forward reshaped image is encoded into an output video signal of the second bit depth.

Wrapped reshaping for codeword augmentation with neighborhood consistency

In a cloud-based system for encoding high dynamic range (HDR) video, each node receives a video segment and bumper frames. Each segment is subdivided into primary scenes and secondary scenes to derive scene-based forward reshaping functions that minimize the amount of reshaping-related metadata when coding the video segment. When a parent scene of a secondary scene is processed by two or more neighboring nodes, initial forward reshaping functions and trim-pass correction parameters are adjusted using reference tone-mapping functions and updated scene-based trim-pass correction parameters.

Trim-pass correction for cloud-based coding of HDR video

In a cloud-based system for encoding high dynamic range (HDR) video, each node receives a video segment and bumper frames. Each segment is subdivided into primary scenes and secondary scenes to derive scene-based forward reshaping functions that minimize the amount of reshaping-related metadata when coding the video segment, while maintaining temporal continuity among scenes processed by multiple nodes. Methods to generate scene-based forward and backward reshaping functions to optimize video coding and improve the coding efficiency of reshaping-related metadata are also examined.

Recursive segment to scene segmentation for cloud-based coding of HDR video

Given an input image in a high dynamic range (HDR) which is mapped to a second image in a second dynamic range using a reshaping function, to improve coding efficiency, a reshaping function generator may adjust the codeword range of the HDR input under certain criteria, such as for noisy HDR images with a relatively-small codeword range. An example of generating a scaler for adjusting the HDR codeword range based on the original codeword range and a metric of the percentage of edge-points in the HDR image is provided. The adjusted reshaping function allows for more efficient rate control during the compression of reshaped images.

Rate-control-aware reshaping in HDR imaging

A method, for generating (a) a forward reshaping function for compressing an input high-dynamic range (HDR) image into a reshaped standard-dynamic-range (SDR) image and (b) a backward reshaping function for decompressing the reshaped SDR image into a reconstructed HDR image, includes (i) optimizing the forward reshaping function to minimize a deviation between the reshaped SDR image and an input SDR image corresponding to the input HDR image, (ii) optimizing the backward reshaping function to minimize a deviation between the reconstructed HDR image and the input HDR image, and (iii) until a termination condition is met, applying a correction to the input SDR image and reiterating, based on the input SDR image as corrected, the steps of optimizing the forward and backward reshaping functions.

Iterative optimization of reshaping functions in single-layer HDR image codec

Input images are received as input to a multi-node system. The input images are divided into segments assigned to respective nodes of the multi-node system. Primary and secondary scenes are identified in the segments to ensure compliance with minimum and average distance constraints. Scene-level forward reshaping mappings are generated for the scenes by a respective node for an assigned segment. Forward reshaped images in the segment are generated by the node using the forward reshaping mappings and encoded into an output video signal, which enables a recipient device to generate reconstructed images and to render display images derived from the reconstructed images on an image display.

Applying minimum and average distance constraint in video streaming

Apparatus and methods for providing software and hardware based solutions to the problem of synthesizing noise for a digital image. According to one aspect, a probability image is generated and noise blocks are randomly placed at locations in the probability image where the locations have probability values that are compared to a threshold criterion, creating a synthesized noise image. Embodiments include generating synthesized film grain images and synthesized digital camera noise images.

Noise synthesis for digital images

Training image pairs comprising training SDR image and corresponding training HDR images are received. Each training image pair in the training image pairs comprises a training SDR image and a corresponding training HDR image. The training SDR image and the corresponding training HDR image in the training image pair depict same visual content but with different luminance dynamic ranges. Training image feature vectors are extracted from training SDR images in the training image pairs. The training image feature vectors are used to train backward reshaping metadata prediction models for predicting operational parameter values of backward reshaping mappings used to backward reshape SDR images into mapped HDR images.

Machine learning based dynamic composing in enhanced standard dynamic range video (SDR+)

A set of tensor-product B-Spline (TPB) basis functions is determined. A set of selected TPB prediction parameters to be used with the set of TPB basis functions for generating predicted image data in mapped images from source image data in source images of a source color grade is generated. The set of selected TPB prediction parameters is generated by minimizing differences between the predicted image data in the mapped images and reference image data in reference images of a reference color grade. The reference images correspond to the source images and depict same visual content as depicted by the source images. The set of selected TPB prediction parameters is encoded in a video signal as a part of image metadata along with the source image data in the source images. The mapped images are caused to be reconstructed and rendered with a recipient device of the video signal.

Growing community of inventors

Santa Clara, CA, United States of America

Harshad Kadu