Google Inc.

Hewlett-packard Development Company, L.p.

Yuxin Liu

Debargha Mukherjee

Yaowu Xu

James Bankoski

Sarah Parker

Yue Chen

Di Chen

Paul Gordon Wilkins

Jingning Han

Adrian Grange

Wei-Ting Lin

Multiple global motion models associated with respective segments of a current frame are decoded from a compressed bitstream. Each global motion model is based on a segmentation of the current frame and represents a respective underlying motion of blocks within a respective segment. Blocks of the current frame are decoded by: for each inter-predicted block of a segment, decoding, form the compressed bitstream, an indication of whether to decode the each inter-predicted block based on a global motion model of the multiple global motion models and associated with the segment, or whether to decode the each inter-predicted block based on a motion vector that is different from the global motion model; and decoding the each inter-predicted block based on the indication.

Segmentation-based parameterized motion models

Generating a compound predictor block for a current block of video includes generating, for the current block, a first predictor block using one of inter-prediction or intra-prediction and generating a second predictor block. The first predictor block includes a first pixel and the second predictor block includes a second pixel that is co-located with the first pixel. A first weight is determined for the first pixel using a difference between a value of the first pixel and a value of the second pixel. A second weight is determined for the second pixel using the first weight. The compound predictor block is generated by combining the first predictor block and the second predictor block. The compound predictor block includes a weighted pixel that is determined using a weighted sum of the first pixel and the second pixel using the first weight and the second weight.

Compound prediction for video coding

Generating a compound predictor block of a current block of video can include generating, for the current block, predictor blocks comprising a first predictor block including first predictor pixels and a second predictor block including second predictor pixels; using at least a subset of the first predictor pixels to determine a first weight for a first predictor pixel of the first predictor pixels; obtaining a second weight for a second predictor pixel of the second predictor pixels, where the second predictor pixel is co-located with the first predictor pixel; and generating the compound predictor block by combining the first predictor block and the second predictor block, where the predictor block includes a weighted pixel that is determined using a weighted sum of the first predictor pixel and the second predictor pixel using the first weight and the second weight, respectively.

Generating a compound predictor block of a current block includes generating, for the current block, predictor blocks including a first predictor block formed of first predictor pixels. Using at least a subset of the first predictor pixels, a first modulation value for modulating a first weight to be applied to a first predictor pixel of the first predictor pixels is determined. The compound predictor block is generated using the first predictor pixel, the first weight, and the first modulation value.

Regions for texture-based coding are identified using a spatial segmentation and a motion flow segmentation. For frames of a group of frames in a video sequence, a frame is segmented using a first classifier into at least one of a texture region or a non-texture region of an image in the frame. Then, the texture regions of the group of frames are segmented using a second classifier into a texture coding region or a non-texture coding region. The second classifier uses motion across the group of frames as input. Each of the classifiers is generated using a machine-learning process. Blocks of the non-texture region and the non-texture coding region of the current frame are coded using a block-based coding technique, while blocks of the texture coding region are coded using a coding technique that is other than the block-based coding technique.

Video compression through motion warping using learning-based motion segmentation

Encoding a group of frames of a video sequence can include determining a stillness of the group of frames, in response to determining that the stillness of the group of frames meets stillness conditions, encoding the group of frames using a coding structure that is a one-layer coding structure, and, in response to determining that the stillness of the group of frames does not meet the stillness conditions, encoding the group of frames using the coding structure that is a multi-layer coding structure. Instructions for decoding a group of frames include instructions to determine, from an encoded bitstream, an indication of a coding structure used to encode the group of frames, receive the group of frames in a coding order of the coding structure, and decode the group of frames using the coding structure. The indication is one of a one-layer coding structure indication or a multi-layer coding structure indication.

Golden-frame group structure design using stillness detection

Coding a current block using multi-level compound predictor is disclosed. An apparatus includes a memory and a processor. The processor is configured to execute instructions stored in the memory to generate four or more prediction blocks; combine a first prediction block and a second prediction block of the four or more prediction blocks to form a first first-level compound prediction block; combine a third prediction block and a fourth prediction block of the four or more prediction blocks to form a second first-level compound prediction block; and combine, to obtain a prediction block for coding the current block, the first first-level compound prediction block and the second first-level compound prediction block.

Multi-level compound prediction

A method includes segmenting the video frame with respect to a reference frame resulting in a segmentation, determining a first motion vector for the current block based on the segmentation, determining a second motion vector for the current block using translational motion compensation, and encoding, for the current block, the one of the first motion vector and the second motion vector corresponding to a smaller error. The segmentation includes a segment containing the current block and a parameterized motion model for the segment. Another method includes identifying a parameterized motion model corresponding to a motion model type, associating the parameterized motion model with a segment of a reference frame, and, in response to determining that the current block is encoded using the parameterized motion model, decoding the current block using the parameterized motion model.

Multi-reference compound prediction using masking for encoding and decoding blocks of video frames is described. A mask for segmenting a current block of a current frame of the video sequence into at least a first segment and a second segment is determined using a first set of reference frames. A first predictor using the first set of reference frames is generated for the first segment of the current block. A second predictor is generated for the second segment of the current block using a second set of reference frames, wherein the first set of reference frames is a subset of the second set of reference frames.

Multi-reference compound prediction of a block using a mask mode

Encoding or decoding blocks of video frames using multiple reference frames with adaptive temporal filtering can include generating one or more candidate reference frames by applying temporal filtering to one or more frames of a video sequence according to relationships between respective ones of the one or more frames and a current frame of the video sequence. A reference frame to use for predicting the current frame can be selected from the one or more candidate reference frames, and a prediction block can be generated using the selected reference frame. During an encoding operation, the prediction block can be used to encode a block of a current frame of the video sequence. During a decoding operation, the prediction block can be used to decode a block of a current frame of the video sequence.

Multi-layer-multi-reference prediction using adaptive temporal filtering

Groups of video frames can be compressed using reversed ordering. For a group of a video sequence having a display order, a determination is made as to whether to arrange the group in a reversed display order before a prediction process of an encoder for the group. Responsive thereto, the group is arranged in the reversed display order as an input order to the prediction process. Otherwise, the group is provided to the prediction process in the display order as the input order. In performing the prediction process, the group, received in the input order, is arranged in a coding order using a group coding structure that defines at least one backward reference frame for inclusion within a set of candidate reference frames. Multiple backward reference frames may also be defined. The group is predicted in the coding order using the set of candidate reference frames.

Compressing groups of video frames using reversed ordering

Coding a current block using multi-level compound predictor is disclosed. A method includes generating a first compound predictor by combining at least two predictor blocks for the current block, generating a second predictor for the current block, and combining the first compound predictor and the second predictor to obtain a prediction block for encoding or decoding the current block. An indicator may be sent from the encoder to the decoder to indicate when multi-level compound prediction is used to encode a block.

Motion vectors are predicted for a current block of a current frame using motion vectors from previous frames in the video stream. Temporal distances between a current frame and the one or more reference frames used to predict a current block and temporal distances between the previous frame and its reference frames are determined. Temporal distances for current frames and previous frames can be combined to weight the motion vectors and improve motion vector prediction.

Motion vector prediction through scaling

Encoding or decoding a video signal includes determining multiple candidate motion vectors for a current block, the candidate motion vectors including a first motion vector used to predict a collocated block within a first frame. A reference frame identifier associated with the first motion vector can be compared to a reference buffer index of a reference frame used to predict the current block of a second frame. On the condition that the reference frame identifier is the same as the reference buffer index, the first motion vector is advanced ahead of remaining ones of the multiple candidate motion vectors.

Motion vector reference selection through reference frame buffer tracking

Techniques for adapting an encoded video signal to a complexity of video encoding may be used to reduce a bit rate of an encoded video signal when the complexity is reduced. Video communication according to the present techniques includes adapting an encoded video signal to an encoding complexity used to derive a set of encoded data from a series of video frames.

Growing community of inventors

Palo Alto, CA, United States of America

Yuxin Liu