Dolby Laboratories Licensing Corporation

Marvell International Limited

Dolby International Ab

University System of Maryland

Dolby Laboratoreis Licensing Corporation

Guan-Ming Su

Peng Yin

Tao Chen

Sheng Qu

Harshad Kadu

Samir N Hulyalkar

Taoran Lu

Walter J Husak

Qian Chen

Sean Thomas McCarthy

Fangjun Pu

Qing Song

Robin Atkins

Hubert Koepfer

Yufei Yuan

Neeraj J Gadgil

Walter Christian Gish

Tsung-Wei Huang

Li Sha

Ching-Han Tsai

Scott J Daly

Yan Ye

Yaojun Luo

Chi-Kuang Chen

Yoon Yung Lee

Amin Kheradmand

Navaneeth Kamballur Kottayil

Cheng-Chi Li

Yuwen He

Gregory John Ward

Jon Scott Miller

David Brooks

Leung Chung Lai

Jan Froehlich

Ye Zhou

Bihan Wen

Wenchi Hsu

Qian Tang

Yee Jin Lee

Arun Raj

Qiuwei Wang

Hsuan-Ting Chou

Sankaranarayanan Piramanayagam

Ashwin Deepak Swaminathan

Timo Kunkel

Yinian Mao

Min Wu

Bongsun Lee

Shan He

Zhen Li

Avinash Laxmisha Varna

Gopi Lakshminarayanan

Hongmei Gou

Anustup Kumar Atanu Choudhury

Qiang Zhu

Hongjie Guan

Klaas Heinrich Schueuer

Per Jonas Andreas Klittmark

Junqiang Lan

Philipp Kraetzer

Yin Peng

Hariharan Ganapathy Kathirvelu

Hossein Talebi Esfandarani

Arjun Arora

Amirhossein Khalilian-gourtani

Ji Qi

Bharath Vishwanath

Hanyang Sun

Janos Horvath

Uwe Michael Kowalik

Arion Neddens

Subhayan Mukherjee

Douglas W Oard

Ralf Willenbacher

Konstantinos Konstaninides

Seyedalireza Golestaneh

Xi Xu

Sean Thomas Mccarthy

An input image represented in an input domain is received from an input video signal. Forward reshaping is performed on the input image to generate a forward reshaped image represented in a reshaped image domain. Non-reshaping encoding operations are performed to encode the reshaped image into an encoded video signal. At least one of the non-reshaping encoding operations is implemented with an ML model that has been previously trained with training images in one or more training datasets in a preceding training stage. A recipient device of the encoded video signal is caused to generate a reconstructed image from the forward reshaped image.

Reshaper for learning based image/video coding

Tensor-Product B-splines (TPB) have been shown to improve video quality when used to represent reshaping functions to map reshaped standard dynamic range content into high dynamic range (HDR) content; however, TPB prediction is computationally intensive and may not be supported by legacy devices. Methods and systems for backwards-compatible signaling of TPB-related metadata and a fast TPB prediction method are presented to overcome both of these limitations. Computation overhead for a TPB-based 3D look-up table is reduced by using temporary two-dimensional arrays. A remapping of the most significant bits of a legacy bit-depth parameter allows for backwards compatibility.

Tensor-product B-spline prediction for HDR video in mobile applications

Methods, systems, and bitstream syntax are described for metadata signaling and film-grain parameter adaptation based on a viewing environment which may differ from a reference environment. Example adaptation models are provided for viewing parameters that include: ambient room illumination, viewing distance, and pixels per inch in a target display. Example systems include a single reference viewing environment model and a multi-reference viewing environment model supporting adaptation of film-grain model parameters via adaptation functions or interpolation.

Film grain parameters adaptation based on viewing environment

Embodiments described herein provide a unified container format for delivering different multimedia applications. One embodiment provides a data structure utilized for implementing a plurality of multimedia applications. The data structure includes a first metadata level including low-level metadata used to perform operations associated with media data in a bitstream. The data structure includes a second metadata level including mid-level metadata used to apply operation metadata to render the media data. The data structure includes a third metadata level including upper-level metadata used to utilize the low-level metadata and the mid-level metadata to deliver the plurality of multimedia applications. The first metadata level further includes synchronization metadata for converting the media data, the low-level metadata, the mid-level metadata, and the upper-level metadata from a first multimedia application of the plurality of multimedia applications to a second multimedia application of the plurality of multimedia applications.

Data structure for multimedia applicatons

Systems and methods for context-based encoding of video data using reshaping algorithms. One method includes receiving the video data, the video data composed of a plurality of image frames, each image frame including a plurality of pixel blocks. The method includes determining, for each pixel block, a luma bin index, determining, for each luma bin, a banding risk value, and determining Gaussian function parameters based on the banding risk value. The method includes generating a differential reshaping function using the Gaussian function parameters, computing a luma-based forward reshaping function based on the differential reshaping function, and generating an output image for each image frame by applying the luma-based forward reshaping function to the respective image frame.

Context-based reshaping algorithms for encoding video data

A first image and a second image of different dynamic ranges are derived from the same source image. Based on a chroma sampling format of the first image, it is determined whether edge preserving filtering is to be used to generate chroma upsampled image data in a reconstructed image. If so, image metadata for performing the edge preserving filtering is generated. The first image, the second image and the image metadata are encoded into an image data container to enable a recipient device to generate the reconstructed image.

Cross-asset guided chroma reformatting for multi-asset imaging format

Given input HDR and SDR images representing the same scene, a prediction model to predict the HDR image from a compressed representation of the input SDR image is generated as follows: a) generate noise data based at least on the characteristics of the HDR image b) generate a noisy SDR image by adding the noise data to the SDR image c) generate an augmented HDR data set and an augmented SDR data set by using the input HDR and SDR images and the noisy SDR image d) generate a prediction model to predict the augmented HDR data set based on the augmented SDR data set and e) solve the prediction model according to a minimization-error criterion to generate a set of prediction parameters to be transmitted to a decoder together with a compressed representation of the input SDR image to reconstruct an approximation of the input HDR image.

Image prediction for HDR imaging in open-loop codecs

Using a standard-based RGB to YCbCr color transform, a new RGB to YCC 3×3 transformation matrix and a 3×1 offset vector are derived under a set of coding-efficiency constraints. The new RGB to YCC 3×3 transform comprises a luminance scaling factor and a 2×2 chroma sub-matrix that preserves the energy of the standard-based RGB to YCbCr transform while maintaining or improving coding efficiency. It also adds support for an authorization or watermarking mechanism in streaming video applications. Examples of using the new color transform using image reshaping are also provided.

Color transformation for HDR video with a coding-efficiency constraint

Methods and systems for generating a set of forward and backward reshaping functions for the efficient coding of high-dynamic range (HDR) images are provided. Given an initial set of forward reshaping functions, output forward reshaping functions are constructed by a) using the forward reshaping functions to generate a first set of corresponding backward reshaping functions b) generating a second set of backward reshaping functions using a multi-segment polynomial representation with a common set of pivot points c) generating an output set of backward reshaping functions by optimizing the polynomial representation of the second set of backward reshaping functions to minimize gap values between consecutive segments and d) using the output set of backward reshaping functions to generate the output set of forward reshaping functions by minimizing the distance between original input HDR codewords and reconstructed HDR codewords.

Reshaping functions for HDR imaging with continuity and reversibility constraints

In an encoder, a high-dynamic range (HDR) image is encoded using a family of local forward reshaping functions selected according to an array of forward mapping indices (FMI) indicating which local forward reshaping function needs to be used for each pixel in the HDR image to generate a reshaped standard dynamic range (SDR) image. A decoder, given the reshaped SDR image, iteratively generates a reconstructed HDR image and estimated reshaped SDR images by adjusting a local FMI array and a local array of backward mapping indices (BMI) until an error metric related to the difference between the local BMI and FMI arrays and the difference between the estimate SDR images and the reshaped SDR image satisfy a convergence criterion. Techniques for generating families of local forward reshaping functions and local backward reshaping functions based on a global forward reshaping function are also presented.

Blind local reshaping in HDR imaging

Guided filtering is applied, with a camera raw image as a guidance image, to a first image to generate an intermediate image. A dynamic range mapping is performed on the intermediate image to generate a second image of a different dynamic range. The second image is used to generate specific local reshaping function index values for selecting specific local reshaping functions. The specific local reshaping functions are applied to the second image to generate a locally reshaped image.

Generating HDR image from corresponding camera raw and SDR images

Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.

Frame-rate scalable video coding

A method for image segmentation includes (a) clustering, based upon k-means clustering, pixels of an image into first clusters, (b) outputting a cluster map of the first clusters (c) re-clustering the pixels into a new plurality of non-disjoint pixel-clusters, and (d) classifying the non-disjoint pixel-clusters in categories, according to a user-indicated classification. Another method for image segmentation includes (a) forming a graph with each node of the graph corresponding to a first respective non-disjoint pixel-cluster of the image and connected to each terminal of the graph and to all other nodes corresponding to other respective non-disjoint pixel-clusters that, in the image, are within a neighborhood of the first respective non-disjoint pixel-cluster, (b) setting weights of connections of the graph according to a user-indicated classification in categories respectively associated with the terminals, and (c) segmenting the image into the categories by cutting the graph based upon the weights.

User-guided image segmentation methods and products

Novel methods and systems for processing a single-layer backward compatible codec with multiple-channel multiple regression coefficients either provided in or pointed to in metadata such that the coefficients have been biased to prevent a shift in neutral colors. Pseudo neutral color patches are used along with a saturation weighting factor to bias the coefficients.

Neutral color preservation for single-layer backward compatible codec

A forward reshaping mapping is generated to map a source image to a corresponding forward reshaped image of a lower dynamic range. The source image is spatially downsampled to generate a resized image into which noise is injected to generate a noise injected image. The forward reshaping mapping is applied to map the noise injected image to generate a noise embedded image of the lower dynamic range. A video signal is encoded with the noise embedded image and delivered to a recipient device for the recipient device to render a display image generated from the noise embedded image.

Adaptive streaming with false contouring alleviation

Growing community of inventors

Fremont, CA, United States of America

Guan-Ming Su