Vid Scale, Inc.

Interdigital Vc Holdings, Inc.

Dolby Laboratories Licensing Corporation

Interdigital Madison Patent Holdings, Sas

Lemon Inc.

Interdigital Patent Holdings, Inc.

Beijing Bytedance Network Technology Co., Ltd.

Bytedance Inc.

Microsoft Corporation

Yuwen He

Yan Ye

Xiaoyu Xiu

Yong He

Jie Dong

Jiancong Luo

Philippe Hanhart

Ralph Neff

Li Feng Zhang

Kai Zhang

Hongbin Liu

George W McClellan

Alexandros Michael Tourapis

Wei Chen

Eun Seok Ryu

Hua Yang

Peng Yin

Yang Wang

Saurav Bandyopadhyay

Chia-Ming Tsai

Yan Zhang

Rahul Vanam

Peshala Vishvajith Pahalawatta

Can Bal

Markus Kunstner

Yue Wang

Athanasios Leontaris

Chun-Chi Chen

Srinivas Gudumasu

Taoran Lu

Samir N Hulyalkar

Jizheng Xu

Shipeng Li

Louis Joseph Kerofsky

Walter Christian Gish

Guan-Ming Su

Na Zhang

Sheng Qu

James E Crenshaw

Yufei Yuan

Christopher Mueller

Eun Ryu

Wei Yu Chen

Siwei Ma

Arash Vosoughi

Ahmed Hamza

Byung Kwan Yi

Tianliang Fu

Junru Li

Fanyi Duanmu

Xueqing Li

Thorsten Laude

Miaohui Wang

Jiun-Yu Kao

Maryam Azimi Hashemi

Enming Luo

Walt J Husak

Hongyu Li

George W Mcclellan

Guobin Shen

Wanyong Cao

Processing video data may include capturing the video data with multiple cameras and stitching the video data together to obtain a 360-degree video. A frame-packed picture may be provided based on the captured and stitched video data. A current sample location may be identified in the frame-packed picture. Whether a neighboring sample location is located outside of a content boundary of the frame-packed picture may be determined. When the neighboring sample location is located outside of the content boundary, a padding sample location may be derived based on at least one circular characteristic of the 360-degree video content and the projection geometry. The 360-degree video content may be processed based on the padding sample location.

360-degree video coding using geometry projection

Systems, methods, and instrumentalities may be provided for discounting reconstructed samples and/or coding information from spatial neighbors across face discontinuities. Whether a current block is located at a face discontinuity may be determined. The face discontinuity may be a face boundary between two or more adjoining blocks that are not spherical neighbors. The coding availability of a neighboring block of the current block may be determined, e.g., based on whether the neighboring block is on the same side of the face discontinuity as the current block. For example, the neighboring block may be determined to be available for decoding the current block if it is on the same side of the face discontinuity as the current block, and unavailable if it is not on the same side of the face discontinuity. The neighboring block may be a spatial neighboring block or a temporal neighboring block.

Handling face discontinuities in 360-degree video coding

Systems, methods, and instrumentalities are disclosed for obtaining coded video data comprising quantized transform coefficients for a plurality of blocks, obtaining a first precision factor associated with a first block for performing at least one decoding function on the first block, obtaining a second precision factor associated with a second block for performing the at least one decoding function on the second block, and performing the at least one decoding function on the quantized transform coefficients for the first block using the first precision factor and on the quantized transform coefficients for the second block using the second precision factor.

Content adaptive transform precision for video coding

Systems and methods are described for reducing the complexity of using bi-directional optical flow (BIO) in video coding. In some embodiments, bit-width reduction steps are introduced in the BIO motion refinement process to reduce the maximum bit-width used for BIO calculations. In some embodiments, simplified interpolation filters are used to generate predicted samples in an extended region around a current coding unit. In some embodiments, different interpolation filters are used for vertical versus horizontal interpolation. In some embodiments, BIO is disabled for coding units with small heights and/or for coding units that are predicted using a sub-block level inter prediction technique, such as advanced temporal motion vector prediction (ATMVP) or affine prediction.

Complexity reduction and bit-width control for bi-directional optical flow

Apparatus and methods for implementing a real-time Versatile Video Coding (VVC) decoder use multiple threads to address the limitation with existing parallelization techniques and fully utilizes the available CPU computation resource without compromising on the coding efficiency. The proposed Multi-threaded (MT) framework uses CTU level parallel processing techniques without compromising on the memory bandwidth. Picture level parallel processing separates the sequence into temporal levels by considering the picture's referencing hierarchy. Embodiments are provided using various optimization techniques to achieve real-time VVC decoding on heterogenous platforms with multi-core CPUs, for those bitstreams generated using a VVC reference encoder with a default configuration.

Methods and apparatus for performing real-time VVC decoding

A device may determine whether to enable or disable bi-directional optical flow (BIO) for a current coding unit (CU) (e.g., block and/or sub-block). Prediction information for the CU may be identified and may include prediction signals associated with a first reference block and a second reference block (e.g., or a first reference sub-block and a second reference sub-block). A prediction difference may be calculated and may be used to determine the similarity between the two prediction signals. The CU may be reconstructed based on the similarity. For example, whether to reconstruct the CU with BIO enabled or BIO disabled may be based on whether the two prediction signals are similar. It may be determined to enable BIO for the CU when the two prediction signals are determined to be dissimilar. For example, the CU may be reconstructed with BIO disabled when the two prediction signals are determined to be similar.

Motion-compensation prediction based on bi-directional optical flow

Systems, methods, and instrumentalities are described herein for calculating local illumination compensation (LIC) parameters for bi-predicted coding unit (CU). The LIC parameters may be used to generate adjusted samples for the current CU and to address local illumination changes that may exist among temporal neighboring pictures. LIC parameters may be calculated based on bi-predicted reference template samples and template samples for a current CU. Bi-predicted reference template samples may be generated based on reference template samples neighboring temporal reference CUs. For example, the bi-predicted reference template samples may be generated based on averaging the reference template samples. The reference template samples may correspond to template samples for the current CU. A CU may be or may include a coding block and/or a sub-block that may be derived by dividing the coding block.

Motion compensated bi-prediction based on local illumination compensation

A filter may be applied to a subset of components associated with a sample in a coding block. The output of the filter may be used to modify values for other component(s). For example, a filter may be applied to a selected (for example, dominant) component(s). The output of the filter may be used to modify a value for one of the other components (for example, non-dominant components). The output of the filter may be used, for example, after a weighting factor is applied to the filter output, to modify a value for another one of the other components. A joint refinement signal may be obtained, for example, as the filtered output signal minus the filter input signal of the selected component(s). A properly weighted version of the joint refinement signal may be applied to modify the other components.

Joint component video frame filtering

A coding device (e.g., that may be or may include encoder and/or decoder) may receive a frame-packed picture of 360-degree video. The coding device may identify a face in the frame-packed picture that the current block belongs to. The coding device may determine that a current block is located at a boundary of the face that the current block belongs to. The coding device may identify multiple spherical neighboring blocks of the current block. The coding device may identify a cross-face boundary neighboring block. The coding device may identify a block in the frame-packed picture that corresponds to the cross-face boundary neighboring block. The coding device may determine whether to use the identified block to code the current block based on availability of the identified block. The coding device may code the current block based on the determination to use the identified block.

360-degree video coding using face continuities

Embodiments of video coding systems and methods are described for reducing coding latency introduced by decoder-side motion vector refinement (DMVR). In one example, two non-refined motion vectors are identified for coding of a first block of samples (e.g. a first coding unit) using bi-prediction. One or both of the non-refined motion vectors are used to predict motion information for a second block of samples (e.g. a second coding unit). The two non-refined motion vectors are refined using DMVR, and the refined motion vectors are used to generate a prediction signal of the first block of samples. Such embodiments allow the second block of samples to be coded substantially in parallel with the first block without waiting for completion of DMVR on the first block. In additional embodiments, optical-flow-based techniques are described for motion vector refinement.

Methods and apparatus for reducing the coding latency of decoder-side motion refinement

Described herein are systems, methods, and instrumentalities associated with video coding. The signaling of certain syntax elements may be moved from a slice header to a picture header and/or a layer access unit delimiter (AUD). The dependency between AUD and one or more parameter sets may be explored. Syntax elements may be signaled to enable wrap-around motion compensation for certain sub-picture(s) and specify wrap-around motion compensation offsets for the sub-picture(s).

Systems and methods for versatile video coding

Systems, methods and instrumentalities are disclosed for adaptively selecting an adaptive loop filter (ALF) procedure for a frame based on which temporal layer the frame is in. ALF procedures may vary in computational complexity. One or more frames including the current frame may be in a temporal layer of a coding scheme. The decoder may determine the current frame's temporal layer level within the coding scheme. The decoder may select an ALF procedure based on the current frame's temporal layer level. If the current frame's temporal layer level is higher within the coding scheme than some other temporal layer levels, an ALF procedure that is less computationally complex may be selected for the current frame. Then the decoder may perform the selected ALF procedure on the current frame.

Methods for simplifying adaptive loop filter in video coding

Methods, devices, apparatus, systems, architectures and interfaces to improve motion vector (MV) refinement based sub-block (SB) level motion compensated prediction are provided. A decoding method includes receiving a bitstream of encoded video data, the bitstream including at least one block of video data including a plurality of SBs; performing a MV derivation, including a decoder based MV (DMVR) process, for at least one SB in the block to generate a refined MV for each SB; performing SB based motion compensation on the at least one sub-block to generate a SB based prediction within each SB; obtaining a spatial gradient for the prediction within each SB; determining a MV offset for each pixel in each SB; obtaining an intensity change in each SB based on the spatial gradients and MV offsets via an optical flow equation; and refining the prediction within each SB based on the obtained intensity changes.

Methods and apparatus for prediction refinement for decoder side motion vector refinement with optical flow

Systems and methods are described for video coding using generalized bi-prediction. In an exemplary embodiment, to code a current block of a video in a bitstream, a first reference block is selected from a first reference picture and a second reference block is selected from a second reference picture. Each reference block is associated with a weight, where the weight may be an arbitrary weight ranging, e.g., between 0 and 1. The current block is predicted using a weighted sum of the reference blocks. The weights may be selected from among a plurality of candidate weights. Candidate weights may be signaled in the bitstream or may be derived implicitly based on a template. Candidate weights may be pruned to avoid out-of-range or substantially duplicate candidate weights. Generalized bi-prediction may additionally be used in frame rate up conversion.

Systems and methods for generalized multi-hypothesis prediction for video coding

A video coding device may be configured to periodically select the frame packing configuration (e.g., face layout and/or face rotations parameters) associated with a RAS. The device may receive a plurality of pictures, which may each comprise a plurality of faces. The pictures may be grouped into a plurality of RASs. The device may select a frame packing configuration with the lowest cost for a first RAS. For example, the cost of a frame packing configuration may be determined based on the first picture of the first RAS. The device may select a frame packing configuration for a second RAS. The frame packing configuration for the first RAS may be different than the frame packing configuration for the second RAS. The frame packing configuration for the first RAS and the frame packing configuration for the second RAS may be signaled in the video bitstream.

Adaptive frame packing for 360-degree video coding

Growing community of inventors

San Diego, CA, United States of America

Yuwen He