Google Inc.

University System of Maryland

Hui Su

Yaowu Xu

Alexander Bokov

Jingning Han

Paul Gordon Wilkins

James Bankoski

Min Wu

Adi Hajj-Ahmad

Debargha Mukherjee

Aki Kuusela

Urvang Joshi

In Suk Chong

Cheng Chen

Yi Luo

Syntax elements are written to a bitstream to designate bit depth precision for palette mode coding of video blocks. During encoding, a bit depth to use for palette mode coding a current block may be based on an input video signal including the current block or based on some change in bit depth precision. A prediction residual for the current block is encoded to a bitstream along with syntax elements indicative of the bit depth used for the palette mode coding of the current block. In particular, the syntax elements include a first element indicating the palette mode coding bit depth used and a second element indicating whether to apply a bit offset to the palette mode coding bit depth. During decoding, values of the syntax elements are read from the bitstream and used to determine a bit depth for palette mode coding the encoded block.

Palette mode coding with designated bit depth precision

A compressed bitstream is configured for decoding by operations that include identifying an adaptive intra-prediction mode indicative of at least a training region or a configuration of neighboring pixel locations. The training region neighbors a block and consists of reconstructed pixels. Filter coefficients used to obtain respective prediction pixels of neighboring pixels within the training region when applied according to the configuration of the neighboring pixels are determined. The filter coefficients minimize a function of differences. Each difference is a respective difference between a pixel in the training region and a prediction of that pixel in the training region. A prediction block is generated for the block by recursive extrapolations that use the filter coefficients by predicting each pixel of the prediction block by applying the filter coefficients to the configuration of neighboring pixels for the pixel being predicted.

Adaptive filter intra prediction modes in image/video compression

For a coding block of an image, a luma prediction block is generated, a luma residual block is generated, a quantized luma block is generated after transforming the luma residual block using a luma transform type, and the quantized luma block is entropy encoded. A chroma prediction block is generated, a chroma residual block is generated, an initial chroma transform type for the chroma residual block is determined as the luma transform type, a quantized chroma block is generated using the chroma residual block transformed by a final chroma transform type, and the quantized chroma block is entropy encoded. When the initial chroma transform type is other than a default transform type, the final chroma transform type is the initial chroma transform type or the default transform type, and quantized coefficients of the quantized chroma block depend upon quantized coefficients of the quantized luma block.

Chroma transform type determination

Generating a prediction block for coding a block includes determining an adaptive intra-prediction mode indicative of at least a training region and a configuration of neighboring pixel locations. The training region neighbors the block and includes a plurality of reconstructed pixels. Filter coefficients are obtained. The filter coefficients are used to obtain respective prediction pixels of neighboring pixels within the training region when applied to defined respective configurations of the neighboring pixels according to the configuration of the neighboring pixels. The filter coefficients minimize a function of differences, each difference being a respective difference between a pixel in the training region and a prediction of that pixel in the training region. The prediction block is generated by recursive extrapolations that use the filter coefficients by predicting each pixel of the prediction block by applying the filter coefficients to the configuration of neighboring pixels for the pixel being predicted.

Adaptive composite intra-prediction may include in response to a determination that a first prediction pixel from a first block immediately adjacent to a first edge of a current block is available for predicting a current pixel of the current block, determining whether a second prediction pixel from a second block immediately adjacent to a second edge of the current block is available for predicting the current pixel, wherein the second edge is opposite the first edge, and, in response to a determination that the second prediction pixel is available, generating a prediction value for the current pixel based on at least one of the first prediction pixel or the second prediction pixel. Adaptive composite intra-prediction may include generating a reconstructed pixel corresponding to the current pixel based on the prediction value, including the reconstructed pixel in the decoded current block, and outputting or storing the decoded current block.

Adaptive composite intra prediction for image and video compression

A processor decodes, from a compressed bitstream, an adaptive intra-prediction mode of a set of adaptive filter modes, the adaptive intra-prediction mode indicating a number of filter coefficients and relative locations with respect to a to-be-predicted pixel of a sub-set of neighboring pixels of the to-be-predicted pixel; determines filter coefficients for generating a prediction block of the block; and generates, by recursive extrapolations that use the filter coefficients and the relative locations, the prediction block. The set of adaptive filter modes includes a first adaptive mode and a second adaptive mode. The first adaptive mode and the second adaptive mode indicate a same number of coefficients. The first adaptive mode indicates a first set of first relative locations of a first sub-set of neighboring pixels that is different from a second set of second relative locations of a second sub-set of neighboring pixels indicated by the second adaptive mode.

Machine learning is used to refine a probability distribution for entropy coding video or image data. A probability distribution is determined for symbols associated with a video block (e.g., quantized transform coefficients, such as during encoding, or syntax elements from a bitstream, such as during decoding), and a set of features is extracted from video data associated with the video block and/or neighbor blocks. The probability distribution and the set of features are then processed using machine learning to produce a refined probability distribution. The video data associated with a video block are entropy coded according to the refined probability distribution. Using machine learning to refine the probability distribution for entropy coding minimizes the cross-entropy loss between the symbols to entropy code and the refined probability distribution.

Entropy coding in image and video compression using machine learning

Generating encoded image data using adjustable per-symbol entropy coding probability updating may include generating a portion of the encoded image data in accordance with a value of a probability update indicator for the portion indicating whether per-symbol entropy coding probability updating is disabled for the portion, and including the value of the probability update indicator and the entropy coded image data in an output bitstream. Generating decoded image data using adjustable per-symbol entropy coding probability updating may include obtaining a value of a probability update indicator for a portion of the decoded image data, the value of the probability update indicator for the portion indicating whether per-symbol entropy coding probability updating is disabled for the portion, and generating decoded image data for the portion in accordance with the value of the probability update indicator for the portion.

Adjustable per-symbol entropy coding probability updating for image and video coding

A method for encoding a block of a video stream includes generating, using pixel values of the block, block features for the block; for each candidate encoding mode of candidate encoding modes, generating, using the block features and the each candidate encoding mode as inputs to a machine-learning module, a respective encoding cost; selecting, based on the respective encoding costs, a predetermined number of the candidate encoding modes; selecting, based on the respective encoding costs of the at least some encoding modes, a best mode for encoding the block; and encoding, in a compressed bitstream, the block using the best mode.

Rate/distortion/RDcost modeling with machine learning

A method for generating a prediction block for coding a block of a frame using intra prediction. The method includes determining, using a training region, filter coefficients for generating the prediction block, the training region neighbors the block and includes a plurality of reconstructed pixels, the filter coefficients minimize a function of differences, each difference being a respective difference between a pixel in the training region and a prediction of that pixel in the training region, and the predictions use the filter coefficients; and generating the prediction block using the determined filter coefficients.

Various systems and methods may benefit from determination of environmental signatures in recordings. For example, such signatures may aid forensic analysis and alignment of media recordings, such as alignment of audio or video recordings. A method can include extracting electric network frequency signals from an image sequence of a video recording or an audio recording. The method can also include synchronizing the video recording or the audio recording with at least one other datum based on the electric network frequency signals.

Techniques to extract ENF signals from video image sequences exploiting the rolling shutter mechanism; and a new video synchronization approach by matching the ENF signals extracted from soundtracks and image sequences

Decoding encoded image data using adjustable per-symbol entropy coding probability updating may include identifying a probability update indicator indicating whether per-symbol entropy coding probability updating is enabled, identifying current entropy coding probabilities, identifying a current entropy coded representation of a current symbol, generating the current symbol by entropy decoding the current entropy coded representation of the current symbol based on the current entropy coding probabilities, in response to a determination that the probability update indicator indicates that per-symbol entropy coding probability updating is enabled for the current portion, generating updated entropy coding probabilities based on the current symbol and the current entropy coding probabilities, generating a decoded current portion using the current symbol, including the decoded current portion in the decoded image data, and outputting the decoded image data.

Video data streams can be encoded and decoded using inter or intra prediction. The blocks of a frame can be processed based on depth, from the lowest level sub-blocks to the highest level large blocks, and divided into groups of blocks that are inter predicted, blocks having sub-blocks that are inter predicted and sub-blocks that are intra predicted, and blocks that are intra predicted. The inter predicted blocks are encoded or decoded first, followed by the inter predicted sub-blocks, the intra predicted sub-blocks, and the intra predicted blocks. Data from the inter predicted blocks and sub-blocks may be used to improve the prediction performance of intra predicted blocks and sub-blocks.

Smart reordering in recursive block partitioning for advanced intra prediction in video coding

Video data streams can be encoded and decoded using inter or intra prediction. The blocks of a frame can be processed based on depth, from the lowest level sub-blocks to the highest level large blocks, and divided into groups of blocks to be inter predicted, blocks having sub-blocks that are to be inter predicted and sub-blocks that are to be intra predicted, and blocks to be intra predicted, and the blocks to be inter predicted are encoded first, the blocks having sub-blocks to be inter predicted and intra predicted encoded second, and the blocks to be intra predicted encoded last. The availability of data from the inter predicted blocks can improve the performance of intra prediction over processing the blocks in the scan order since more pixel data is available for intra prediction of some blocks.

Growing community of inventors

Sunnyvale, CA, United States of America

Hui Su