Dolby Laboratories Licensing Corporation

Micronas Usa, Inc.

Sheng Qu

Guan-Ming Su

Samir N Hulyalkar

Hubert Koepfer

Yufei Yuan

Peng Yin

Walter Christian Gish

Tao Chen

Qian Chen

Yan Ye

Robin Atkins

Yuwen He

Raymond Yeung

David Brooks

Scott J Daly

Gregory John Ward

Harshad Kadu

Zhen Li

Neeraj J Gadgil

Gopi Lakshminarayanan

Hongjun Yuan

Per Jonas Andreas Klittmark

Gopi Lakshminarayan

Yin Peng

Hariharan Ganapathy Kathirvelu

Ailan Hu

Xi Xu

Daniel W Meyer

Tensor-Product B-splines (TPB) have been shown to improve video quality when used to represent reshaping functions to map reshaped standard dynamic range content into high dynamic range (HDR) content; however, TPB prediction is computationally intensive and may not be supported by legacy devices. Methods and systems for backwards-compatible signaling of TPB-related metadata and a fast TPB prediction method are presented to overcome both of these limitations. Computation overhead for a TPB-based 3D look-up table is reduced by using temporary two-dimensional arrays. A remapping of the most significant bits of a legacy bit-depth parameter allows for backwards compatibility.

Tensor-product B-spline prediction for HDR video in mobile applications

Embodiments described herein provide a unified container format for delivering different multimedia applications. One embodiment provides a data structure utilized for implementing a plurality of multimedia applications. The data structure includes a first metadata level including low-level metadata used to perform operations associated with media data in a bitstream. The data structure includes a second metadata level including mid-level metadata used to apply operation metadata to render the media data. The data structure includes a third metadata level including upper-level metadata used to utilize the low-level metadata and the mid-level metadata to deliver the plurality of multimedia applications. The first metadata level further includes synchronization metadata for converting the media data, the low-level metadata, the mid-level metadata, and the upper-level metadata from a first multimedia application of the plurality of multimedia applications to a second multimedia application of the plurality of multimedia applications.

Data structure for multimedia applicatons

Sequence-level parameters are generated for an image frame sequence including sequence-level indicators for indicating metadata types present for each image frame in the sequence of image frames. Frame-present parameters are generated for a specific image frame in the sequence including frame-present indicators corresponding to the metadata types as indicated in the sequence-level parameters. The frame-present indicators identify first metadata types for which metadata parameter values are to be encoded in a coded bitstream as metadata payloads. The image frame sequence, the sequence-level parameters, the frame-present parameters and the metadata payloads are encoded in the coded bitstream. A recipient device can generate, from the specific image frame based partly on the metadata parameter values determined for the first metadata types, a target display image for a target display.

Backward compatible display management metadata compression

Coding syntaxes in compliance with same or different VDR specifications may be signaled by upstream coding devices such as VDR encoders to downstream coding devices such as VDR decoders in a common vehicle in the form of RPU data units. VDR coding operations and operational parameters may be specified as sequence level, frame level, or partition level syntax elements in a coding syntax. Syntax elements in a coding syntax may be coded directly in one or more current RPU data units under a current RPU ID, predicted from other partitions/segments/ranges previously sent with the same current RPU ID, or predicted from other frame level or sequence level syntax elements previously sent with a previous RPU ID. A downstream device may perform decoding operations on multi-layered input image data based on received coding syntaxes to construct VDR images.

Specifying visual dynamic range coding operations and parameters

In a method to code and transmit scalable HDR video signals, HDR signals are processed and encoded in the IPT-PQ color space to generate a base layer at reduced spatial resolution and/or dynamic range, and an enhancement layer with a residual signal. A signal reshaping block before the base layer encoder allows for improved coding of HDR signals using a reduced bit depth. A decoder can use a BL decoder and backward reshaping to generate a decoded BL HDR signal at a reduced dynamic range and/or spatial resolution, or it can combine the decoded BL HDR signal and the EL stream to generate a decoded HDR signal at full dynamic range and full resolution.

Video coding and delivery with both spatial and dynamic range scalability

Methods and systems for generating and applying scene-stable metadata for a video data stream are disclosed herein. A video data stream is divided or partitioned into scenes and a first set of metadata may be generated for a given scene of video data. The first set of metadata may be any known metadata as a desired function of video content (e.g., luminance). The first set of metadata may be generated on a frame-by-frame basis. In one example, scene-stable metadata may be generated that may be different from the first set of metadata for the scene. The scene-stable metadata may be generated by monitoring a desired feature with the scene and may be used to keep the desired feature within an acceptable range of values. This may help to avoid noticeable and possibly objectionably visual artifacts upon rendering the video data.

System and methods for generating scene stabilized metadata

In layered Visual Dynamic range (VDR) coding, inter-layer prediction requires several color-format transformations between the input VDR and Standard Dynamic Range (SDR) signals. Coding and decoding architectures are presented wherein inter-layer prediction is performed in the SDR-based color format, thus reducing computational complexity in both the encoder and the decoder, without compromising coding efficiency or coding quality.

Efficient architecture for layered VDR coding

Inter-color image prediction is based on multi-channel multiple regression (MMR) models. Image prediction is applied to the efficient coding of images and video signals of high dynamic range. MMR models may include first order parameters, second order parameters, and cross-pixel parameters. MMR models using extension parameters incorporating neighbor pixel relations are also presented. Using minimum means-square error criteria, closed form solutions for the prediction parameters are presented for a variety of MMR models.

Multiple color channel multiple regression predictor

In a decoder, a processor extracts a control map of false contour filtering from a part of a multi-layer video signal that includes a low dynamic range image mapped from an original high-dynamic range (HDR) image. It determines one or more filter parameters for a sparse finite-impulse-response (FIR) filter, where the one or more filter parameters relate to at least in part on the control map of false contour filtering and a predicted image predicted from the low dynamic range image. It applies the sparse FIR filter to filter pixel values in a portion of the predicted image based at least in part on the control map of false contour filtering, and it reconstructs a version of the original HDR image based at least in part on the portion of the predicted image as filtered by the FIR filter.

Hardware efficient sparse FIR filtering in layered video coding

Systems and methods are disclosed for filtering metadata to be used in display management. Given an input video stream and input metadata comprising at least one of minimum, average, or maximum luminance values of the video frames in the video stream, values of a function of the input metadata are filtered using a temporal filter to generate filtered metadata, wherein the filtering is based only on metadata for input frames in the same scene. Methods for temporal filtering based on an exponential moving average filter or a look-ahead sliding window filter are presented, including methods for scene-change detection using the input metadata.

Metadata filtering for display mapping for high dynamic range images

In an embodiment, a media source combines reference code values and mapping function parameters for mapping functions into video frames originally designated to carry pixel values. The video frames are delivered to a downstream device such as a media sink in an encoded video signal. The media sink extracts the mapping function parameters for the mapping functions from the encoded video signal and applies the mapping functions as a part of display management operations to map the reference code values to the mapped pixel values appropriate for the media sink. The mapped pixel values can be used to render images as represented by the reference code values.

Transmitting display management metadata over HDMI

Techniques use multiple lower bit depth (e.g., 8 bits) codecs to provide higher bit depth (e.g., 12+ bits) high dynamic range images from an upstream device to a downstream device. Multiple layers comprising a base layer and one or more enhancement layers may be used to carry video signals comprising image data compressed by lower bit depth encoders to a downstream device, wherein the base layer cannot be decoded and viewed on its own. Lower bit depth input image data to base layer processing may be generated from higher bit depth high dynamic range input image data via advanced quantization to minimize the volume of image data to be carried by enhancement layer video signals. The image data in the enhancement layer video signals may comprise residual values, quantization parameters, and mapping parameters based in part on a prediction method corresponding to a specific method used in the advanced quantization. Adaptive dynamic range adaptation techniques take into consideration special transition effects, such as fade-in and fade-outs, for improved coding performance.

Layer decomposition in hierarchical VDR coding

In an embodiment, a source video signal comprising reference code values and mapping function parameters for one or more mapping functions to map the reference code values to device-specific pixel values. The reference code values and the mapping function parameters are combined into first video frames in a first video signal format. The mapping function parameters represent DM metadata carried in the first video frames. Second video frames in a second video signal format are generated based on the first video frames in the first video signal format and a mapping between the first video signal format and the second video signal format. The second video frames are sent to a video sink device through the video link in an encoded video signal of the second video signal format.

Growing community of inventors

San Jose, CA, United States of America

Sheng Qu