Comcast Cable Communications, LLC

Fraunhofer-gesellschaft Zur Foerderung Der Angewandten Forschung E.v.

Dan Grois

Alexander Giladi

Yasser Fouad Syed

Ali C Begen

Thomas Wiegand

Detlev Marpe

Heiko Schwarz

Benjamin Bross

Stuart Kurkowski

Georg Maier

Remco C Veltkamp

Methods, systems, apparatuses for selecting content applications are described. A manifest file associated with a channel may include content item identifiers associated with content items and content application identifiers associated with content applications that can play the content items. A computing device such as a set-top box may determine one or more content applications to play one or more content items based on the content item identifiers and the content application identifiers in the manifest file.

Methods and apparatuses for selecting content applications

Methods, systems, and apparatuses are described for encoding video. Video content to be encoded and sent to a computing device may be downscaled into one or more layers. The one or more layers may represent one or more versions of the video content such as one or more versions encoded at different resolutions. The residuals between each layer and the base layer may be upscaled so that one or more parameters associated with optimizing the encoding of the one or more layers may be determined by one or more neural networks based on the downscaling and upscaling process. The residuals between each layer, the one or more parameters, and the base layer may be encoded and sent to a computing device for decoding and playback of the video content using any of the versions of the video content.

Methods, systems, and apparatuses for content-adaptive multi-layer coding based on neural networks

A first quantization value for encoding at least one frame of a content item may be determined based at least on a predetermined bitrate and a point in the content item associated with a scene change. A first duration associated with a first portion of the content item may be determined. The first portion of the content item may comprise the at least one frame and may be associated with the first quantization value. A second quantization value for encoding at least another frame of the content item may be determined based at least on the predetermined bitrate. A second duration associated with a second portion of the content item may be determined. The second portion of the content item may comprise the at least another frame and may be associated with the second quantization value.

Processing video using masking windows

Systems, methods, and apparatuses are described for processing video. A method may comprise receiving video content comprising a plurality of frames and determining a splitting parameter of a partitioning of at least a portion of a first frame of the plurality of frames. Deblock filtering may be implemented on at least the first frame of the plurality of frames based at least on the splitting parameter.

Systems, methods, and apparatuses for processing video

An encoder may determine a plurality of coding units associated with a frame of a media file and a plurality of prediction units associated with the frame of the media file. The encoder may determine, based on the plurality of coding units associated with the frame and the plurality of prediction units associated with the frame, and based on a training of the encoder using one or more neural networks, that a particular region of the frame can be encoded using one or more encoding characteristics that are different than the encoding characteristics of one or more other particular regions of the frame. The encoder may allocate one or more encoding resources to the particular region of the frame based on the one or more encoding characteristics of the particular region of the frame in order to reduce the overall media bitrate.

Processing media using neural networks

Methods and systems for in-loop filtering may comprise receiving a video comprising at least one frame. The frame may comprise at least one block of pixels. A content complexity of the block of pixels may be determined. A viewing characteristic of the video may be determined. A number of borders to be filtered may be determined based on at least one of the content complexity or the viewing characteristic. A deblocking filter strength may be determined based on at least one of the content complexity or the viewing characteristic. The number of borders of the block of pixels may be filtered according to the deblocking filter strength.

Systems and methods for deblocking filtering

An encoder (e.g., a Versatile Video Coding (VVC) encoder, etc.) may make real-time encoding decisions to vary a bitrate of a content item (e.g., high-resolution video, streaming content, a movie, a show/program, etc.) by upsampling and/or downsampling portions (e.g., frames/slices, group of pictures (GOP), coding units (CU), coding tree units (CTU), etc.) based on a cost function that utilizes a versatile Lagrangian multiplier (denoted as λto denote the versatility). The versatile Lagrangian multiplier may not only account for the effects of quantization (e.g., a quantization parameter (QP), etc.) on portions of the content item, instead, the versatile Lagrangian multiplier may also be based on adjustment parameters associated with content resolution and/or playback quality of experience (QoE). The versatile Lagrangian multiplier enables a substantial decrease in transmission bitrate for portions of a content item without reducing the visual presentation quality and keeping an end-user quality of experience (QoE) at an optimal level.

Method and systems for optimized content encoding

Systems, methods, and apparatuses are described for processing video. Video content comprising a plurality of frames may be received. A visual element of a first frame of the plurality of frames positioned in an oblique direction relative to one or more of a first axis and a second axis orthogonal to the first axis may be determined. One or more regions associated with the first frame and comprising the visual element may be determined. One or more encoded regions of the first frame may be generated based on partitioning the one or more regions comprising the visual element.

A spatial complexity and a temporal complexity associated with one or more frames of media content may be determined. Based on the spatial complexity and the temporal complexity of the media content, a Group of Picture (GOP) size for the one or more frames of the media content may be determined. The GOP size may be inversely proportional to the spatial complexity and the temporal complexity of the one or more frames of media content. Certain frames of the media content may be arranged in a different GOP size as compared to one or more other frames of the media content. By varying the GOP size of the plurality of frames of the media content, the bitrate required to transmit the media content may be decreased without decreasing or substantially decreasing the overall quality of the media content.

Processing media by adaptive group of pictures structuring

Methods, systems, and apparatuses are described for streaming content. The embodiments described herein may apply to ABR streaming and output an optimized ABR ladder enabling a playback device to maximize the perceived video quality of received content, provide substantially constant visual quality, and/or minimize rebuffering content. The playback device may request an ABR representation associated with an ABR ladder that has been optimized based on one or more neural networks. The one or more neural networks may be pre-trained based on one or more properties and characteristics/features of the video content. The one or more neural networks may process the video content to dynamically determine parameters such as the bitrates of each ABR representation for the content. The parameters may be used to output the encoded ABR representations.

Methods, systems, and apparatuses for adaptive bitrate ladder construction based on dynamically adjustable neural networks

Methods, systems, and apparatuses for adaptive processing of video content to remove noise, such as film grain noise, without substantially affecting visual presentation quality are described herein. A computing device may determine a plurality of film grain parameters associated with film grain noise present within one or more portions of a content item. The computing device may determine at least one encoding parameter based on the plurality of film grain parameters. The computing device may encode the content item based on the at least one encoding parameter. The computing device may send an encoding message to at least one user device/client device, which may in turn use the encoding message to decode the content item.

Methods, systems, and apparatuses for adaptive processing of video content with film grain

Methods and systems for improved content encoding are described herein. An encoder may allocate a time budget for encoding each frame of a content item. When the encoder takes longer than the allocated time budget to encode at least a portion of a given frame(s) of the content item at a first resolution, the encoder may take a number of actions in order to allow the encoder to 'catch up,' such as encoding frames at a second resolution. Once the encoder catches up and the allocated time budget is not being exceeded, the encoder may resume encoding frames of the content item at the first resolution.

Methods and systems for improved content encoding

Systems and methods are described herein for processing video. An encoder implementing the systems and methods described herein may receive video data comprising a plurality of frames and may partition each frame of the plurality of frames into a plurality of coding units. The encoder may then partition a coding unit into two or more prediction units. The encoder may determine, based on one or more coding parameters, a target bit rate, and characteristics of a human visual system (HVS), a coding mode for each of the two or more prediction units to minimize distortion in the encoded bitstream. The encoder may then determine a residual signal comprising a difference between each of the two or more prediction units and each of one or more corresponding prediction areas in a previously encoded frame and then generate an encoded bitstream comprising the residual signal.

Growing community of inventors

Beer-Sheva, Israel

Dan Grois