Vid Scale, Inc.

Amazon Technologies, Inc.

Interdigital Vc Holdings, Inc.

Rahul Vanam

Yuriy Reznik

Yuwen He

Yan Ye

Gregory S Sternberg

Eduardo Asbun

Zhifeng Chen

Liangping Ma

Wei Yu Chen

Sriram Sethuraman

Louis Joseph Kerofsky

Weimin Liu

Robert L Olesen

Wei Chen

Ralph Neff

Charles Benjamin Waggoner

Hua Yang

Dharm Veer

Systems, methods and instrumentalities are disclosed for adaptively selecting an adaptive loop filter (ALF) procedure for a frame based on which temporal layer the frame is in. ALF procedures may vary in computational complexity. One or more frames including the current frame may be in a temporal layer of a coding scheme. The decoder may determine the current frame's temporal layer level within the coding scheme. The decoder may select an ALF procedure based on the current frame's temporal layer level. If the current frame's temporal layer level is higher within the coding scheme than some other temporal layer levels, an ALF procedure that is less computationally complex may be selected for the current frame. Then the decoder may perform the selected ALF procedure on the current frame.

Methods for simplifying adaptive loop filter in video coding

Techniques are described for pre-processing frames to apply a temporal filter. A P-frame may be temporally filtered based on neighboring reference frames. The temporal filter may be applied to blocks of the P-frame to improve alignment with dependent frames that are encoded based on blocks of the P-frame.

Video pre-processing using encoder-aware motion compensated residual reduction

Intra sub-partitions (ISP) may be enabled for a current block, for example, based on an ISP indication. The block may be partitioned into multiple sub-partitions, and a sub-partition may belong to a prediction unit (PU). A sub-partition width for the current block and a minimum prediction block width may be obtained. A PU corresponding to a current sub-partition may be determined based on the sub-partition width and the minimum prediction block width. For example, when the sub-partition width is less than the minimum prediction block width, the PU may include multiple sub-partitions. In examples, the minimum prediction block width may be four samples. Reference samples may be determined, and the PU may be predicted using the reference samples.

Intra sub-partitions in video coding

A video encoding device (e.g., a wireless transmit/receive unit (WTRU)) may transmit an encoded frame with a frame sequence number using a transmission protocol. The video encoding device, an application on the video encoding device, and/or a protocol layer on the encoding device may detect a packet loss by receiving an error notification. The packet loss may be detected at the MAC layer. The packet loss may be signaled using spoofed packets, such as a spoofed NAM packet, a spoofed XR packet, or a spoofed ACK packet. A lost packet may be retransmitted at the MAC layer (e.g., by the encoding device or another device on the wireless path). Packet loss detection may be performed in uplink operations and/or downlink operations, and/or may be performed in video gaining applications via the cloud. The video encoding device may generate and send a second encoded frame based on the error notification.

Early packet loss detection and feedback

Embodiments include systems, methods, and computer-readable media for optimized reduced bitrate encoding for text-based content in video frames. Example methods may include determining that a first segment of video content includes a content scene, determining that a second segment of the video content includes text, and determining a first encoder configuration to encode the first segment of video content, where the first encoder configuration includes a first encoding parameter setting. Example methods may include determining a second encoder configuration to encode the second segment of the video content, where the second encoder configuration includes a second encoding parameter setting, encoding the first segment using the first encoder configuration, and encoding the second segment using the second encoder configuration. The first segment may be encoded at a first bitrate that is greater than a second bitrate at which the second segment is encoded.

Optimized reduced bitrate encoding for titles and credits in video content

A video encoding device (e.g., a wireless transmit/receive unit (WTRU)) may transmit an encoded frame with a frame sequence number using a transmission protocol. The video encoding device, an application on the video encoding device, and/or a protocol layer on the encoding device may detect a packet loss by receiving an error notification. The packet loss may be detected at the MAC layer. The packet loss may be signaled using spoofed packets, such as a spoofed NACK packet, a spoofed XR packet, or a spoofed ACK packet. A lost packet may be retransmitted at the MAC layer (e.g., by the encoding device or another device on the wireless path). Packet loss detection may be performed in uplink operations and/or downlink operations, and/or may be performed in video gaining applications via the cloud. The video encoding device may generate and send a second encoded frame based on the error notification.

Intra planar approach(es) may be used to predict a pixel(s) in a current block. The current block may be associated with a reconstructed left reference line, a reconstructed top reference line, and an non-reconstructed reference line to be predicted. The reconstructed reference lines may have been decoded and may be available. The non-reconstructed reference lines to be predicted may include an non-reconstructed right and/or an non-reconstructed bottom reference lines. A pivot reference pixel may be identified and may be located on an extension of the reconstructed left and/or top reference lines. A reference pixel may be determined and may be located on the reconstructed top and/or left reference lines. Pixels on the non-reconstructed reference line(s) may be predicted based on the pivot reference pixel and the reference pixel. Pixels of the current block may be predicted using the predicted pixels on the right and the bottom reference lines.

Prediction approaches for intra planar coding

Quality-based optimizations of a delivery process of streaming content may be enabled. The optimization may take the form of quality-based switching. To enable quality-based switching in a streaming client, the client may have access to information about the quality of an encoded segment and/or sub-segment. Quality-related information may include any number of added quality metrics relating to an encoded segment and/or sub-segment of an encoded video stream. The addition of quality-related information may be accomplished by including the quality-related information in a manifest file, including the quality-related information in segment indices stored in a segment index file, and/or providing additional files with quality-related segment information and providing a link to the information from an MPD file. Upon receiving the quality-related information, the client may request and receive a stream that has a lower bitrate, thereby saving bandwidth while retaining quality of the streaming content.

Quality-driven streaming

Prediction systems and methods for video coding are described based on nearest neighboring pixels. In exemplary embodiments, to code a first pixel, a plurality of neighboring pixels of the first pixel are reconstructed. The coefficients of a filter such as a Wiener filter are derived based on the reconstructed neighboring pixels. The Wiener filter is applied to the reconstructed neighboring pixels to predict the first pixel. The coefficients of the Wiener filter may be derived on a pixel-by-pixel or a block-by-block basis. The reconstructed pixels may be pixels in the same picture (for intra prediction) or in a reference picture (for inter prediction). In some embodiments, the residuals of the prediction are encoded using RDPCM. In some embodiments, the residuals may be predicted using a Wiener filter.

Prediction systems and methods for video coding based on filtering nearest neighboring pixels

Visual information may be delivered to streaming-capable devices in a viewing environment, such as a home environment or a commercial environment. The visual information can be adapted to user behavior and/or viewing conditions in such a way as to deliver a satisfactory user experience while conserving network resources, such as bandwidth and/or capacity. Viewing distance and/or ambient light, which may affect viewing conditions in a viewing environment, may be estimated. Bandwidth may be reduced by eliminating details that may not be perceived by the user in the estimated viewing conditions (e.g., by determining a spatial resolution (e.g. a maximum spatial resolution) perceptible under the viewing conditions and not exceeding that spatial resolution).

Viewing conditions estimation for adaptive delivery of visual information in a viewing environment

A perceptual filter may be implemented to filter one or more spatial frequencies from a video signal that are below a contrast sensitivity limit of a viewer of the video signal. The perceptual filter may be configured to adapt one or more perceptual filter parameters on a pixel-basis based on, for example, content, viewing distance, display density, contrast ratio, display luminance, background luminance, and/or age of the viewer. A spatial cutoff frequency of the perceptual filter may be mapped to a contrast sensitivity. The perceptual filter may be used as a preprocessing step for a video encoder so as to lower an encoded bitrate. Temporal filtering of the video frames may be used to maintain continuity of a spatial cutoff frequency to ensure the perceptual filtering effects are not identified as motion by a video encoder, and the temporal filtering may be restricted to static areas of a frame.

Perceptual preprocessing filter for viewing-conditions-aware video coding

Systems, methods, and instrumentalities are disclosed for denoising high dynamic range video using a temporal filter. A frame of an uncompressed video stream may be received. The frame may include a plurality of pixels. It may be determined that a chroma component of a pixel of the plurality of pixels belongs to a predefined region. The region may be predefined on the basis of one or more chroma components. The predefined region may be a CbCr space. The chroma component of the pixel may be compared with a chroma component of a co-located pixel to determine if the chroma component of the co-located pixel belongs to the predefined region. The pixel may be filtered using a denoising filter. The denoising filter may be a temporal filter. The denoising filter may be a Crawford filter. The uncompressed video stream including the filtered pixel may be encoded.

Growing community of inventors

San Diego, CA, United States of America

Rahul Vanam