Qualcomm Incorporated

Hilmi Enes Egilmez

Marta Karczewicz

Vadim Seregin

Amir Said

Muhammed Zeyd Coban

Alican Nalci

Nan Hu

Ankitesh Kumar Singh

Li Feng Zhang

Wei-Jung Chien

Geert Van Der Auwera

Yung-Hsuan Chao

Xin Zhao

Adarsh Krishnan Ramasubramonian

Bappaditya Ray

Cheng-Teh Hsieh

Luong Pham Van

Louis Joseph Kerofsky

Akshay Gadde

A video decoder determines, based on a block size of a current block and a low-frequency non-separable transform (LFNST) syntax element, a zero-out pattern of normatively defined zero-coefficients. The LFNST syntax element is signaled at a transform unit (TU) level. Additionally, the video decoder determines transform coefficients of the current block. The transform coefficients of the current block include transform coefficients in an LFNST region of the current block and transform coefficients outside the LFNST region of the current block. As part of determining the transform coefficients of the current block, the video decoder applies an inverse LFNST to determine values of one or more transform coefficients in the LFNST region of the current block. The video decoder also determines that transform coefficients of the current block in a region of the current block defined by the zero-out pattern are equal to 0.

Transform and last significant coefficient position signaling for low-frequency non-separable transform in video coding

A device may perform a first prediction process for a first block of video data to produce a first residual. The device may apply a first transform process to the first residual to generate first transform coefficients for the first block of video data and encode the first transform coefficients. The device may perform a second prediction process for a second block of video data to produce a second residual. The device may determine that a second transform process, which includes the first transform process and at least one of a pre-adjustment operation or a post-adjustment operation, is to be applied to the second residual. The device may apply the first transform process and the pre- or post-adjustment operation to the second residual to generate second transform coefficients for the second block. The coding device may code the first and second transform coefficients.

Multiple transforms adjustment stages for video coding

Techniques are described herein for processing video data using a neural network system. For instance, a process can include generating, by a first convolutional layer of an encoder sub-network of the neural network system, output values associated with a luminance channel of a frame. The process can include generating, by a second convolutional layer of the encoder sub-network, output values associated with at least one chrominance channel of the frame. The process can include generating, by a third convolutional layer based on the output values associated with the luminance channel of the frame and the output values associated with the at least one chrominance channel of the frame, a combined representation of the frame. The process can further include generating encoded video data based on the combined representation of the frame.

Front-end architecture for neural network based video coding

Systems and techniques are described herein for processing video data. In some aspects, a method can include obtain, by a machine learning system, input video data. The input video data includes one or more luminance components for a current frame. The method can include determining, by the machine learning system, motion information for the luminance component(s) of the current frame and motion information for one or more chrominance components of the current frame using the luminance component(s) for the current frame. In some cases, the method can include determining the motion information for the luminance component(s) based on the luma component(s) of the current frame and at least one reconstructed luma component of a previous frame. In some cases, the method can further include determining the motion information for the chrominance component(s) of the current frame using the motion information determined for the luminance component(s) of the current frame.

Machine learning based flow determination for video coding

An example method of decoding video data that includes receiving one or more syntax elements of the video data indicative of whether a first type of coding scheme or a second type of coding scheme is applied to residual values of a block of video data coded with transform skip, wherein the residual values are indicative of a difference between the block and a prediction block, and wherein, in transform skip, the residual values are not transformed from a sample domain to a frequency domain. The method includes determining a type of coding scheme to apply to the residual values based on the one or more syntax elements, determining the residual values based on the determined type of coding scheme, and reconstructing the block based on the determined residual values and the prediction block.

Signaling coding scheme for residual values in transform skip for video coding

An example method includes determining a color component of a unit of video data; determining, based at least on the color component, a context for context-adaptive binary arithmetic coding (CABAC) a syntax element that specifies a value of a low-frequency non-separable transform (LFNST) index for the unit of video data; CABAC decoding, based on the determined context and via a syntax structure for the unit of video data, the syntax element that specifies the value of the LFNST index for the unit of video data; and inverse-transforming, based on a transform indicated by the value of the LFNST index, transform coefficients of the unit of video data.

Context modeling for low-frequency non-separable transformation signaling for video coding

A video decoder configured to set a context index variable to a first value, wherein the first value for the context index variable is associated with a first context; context decode a first bin for a syntax element indicating a transform using the first context; determine a new value for the context index variable based on a value of the first bin; context decode a second bin for the syntax element indicating the transform using a context associated with the new value; determine an inverse transform from a set of inverse transform candidates based on the first bin and the second bin; and apply the inverse transform to a set of coefficients to determine a block of residual data.

Context-based transform index signaling in video coding

A method of decoding video data includes determining an intra prediction mode from a plurality of intra prediction modes for a current block of the video data, determining a low frequency non-separable transform (LFNST) kernel from a plurality of LFNST kernels for the current block based on the determined intra prediction mode, wherein at least one LFNST kernel of the plurality of LFNST kernels is associated with at least two different intra prediction modes of the plurality of intra prediction modes, applying an inverse of the determined LFNST kernel to coefficient values generated from a transform unit (TU) of the current block to generate intermediate values, applying an inverse primary transform on the intermediate values to generate residual data, and reconstructing the current block based on the residual data.

Low frequency non-separable transform for video coding

A video decoder can be configured to determine a number of allowed non-zero coefficients for a block of video data based on a size of the block; obtain a set of dequantized coefficients for the block, wherein the set of dequantized coefficients comprises a first subset of dequantized coefficients that includes non-zero dequantized coefficients and a second subset of dequantized coefficients that includes all zero coefficients, wherein a number of coefficients in the first subset of dequantized coefficients is equal to the number of allowed non-zero coefficients for the block of video data; apply an inverse low-frequency non-separable transform (LFNST) to the first subset of dequantized coefficients to determine a first intermediate subset of coefficients; and apply an inverse separable transform to the first intermediate subset of coefficients and at least a portion of the second subset of coefficients to determine a block of reconstructed residual values.

Extended low-frequency non-separable transform (LFNST) designs with worst-case complexity handling

Example techniques are described for coding video data by obtaining a block of video data, obtaining an adaptive parameter set, determining a set of adaptive loop filter parameters for a plurality of filters for the block of video data based on the adaptive parameter set, wherein a plurality of adaptive loop parameters of the set of adaptive loop filter parameters are signaled using the same signaling parameter for each of the plurality of filters of the adaptive parameter set, and coding the block of video data using the set of adaptive loop filter parameters. The example techniques can be performed as part of an encoding or decoding process and/or by an encoder or a decoder.

Adaptive loop filter signalling

A video decoder may be configured to receive a block of video data, and determine a value of a multiple transform set index for the block of video data based on a presence of non-zero transform coefficients in the block of video data other than a DC coefficient. The video decoder may then apply a transform to the block of video data based on the determined value of the multiple transform set index.

Multiple transform set signaling for video coding

An example device for coding video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: code a first codeword representing a selected transform scheme of a set of transform candidates of a multiple transform selection (MTS) scheme for a current block of video data, the selected transform scheme being a secondary transform of a set of available secondary transforms to be applied in addition to a primary transform; code a second codeword representing the secondary transform from the set of available secondary transforms; and apply the primary transform and the secondary transform during coding of residual data for the current block. The second codeword may be a value for a low-frequency non-separable transform (LFNST) syntax element.

Extended multiple transform selection for video coding

A method of decoding video data includes receiving encoded data for a current block and decoding N bins for a low-frequency non-separable transform (LFNST) index from the encoded data. The N bins comprises a first bin and a second bin. Decoding the N bins comprises context decoding each bin of the N bins. The method further includes determining the LFNST index using the N bins and decoding the encoded data to generate transform coefficients. The method further includes applying an inverse LFNST to the transform coefficients using the LFNST index to produce a residual block for the current block and reconstructing the current block of the video data using the residual block and a prediction block for the current block.

Growing community of inventors

San Diego, CA, United States of America

Hilmi Enes Egilmez