Disney Enterprises, Inc.

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Beijing Yojaja Software Technology Development Co., Ltd.

Eth Zurich

Yuanyi Xue

Scott C Labrozzi

Christopher Richard Schroers

Abdelaziz Djelouah

Michael J Bracco

Roberto Gerson De Albuquerque Azevedo

Wenhao Zhang

Erika Varis Doggett

Jared McPhillen

Leonhard Markus Helminger

Chetan K Mathur

Joaquim Campos

Xuchang Huangfu

Yang Zhang

Nicholas David Gregory

Chen Liu

Tunc Ozan Aydin

Xiaobo Liu

Lemei Huang

James D Zimmerman

Erika Elizabeth Varis Doggett

Simon Meierhans

Mugdha Oltikar

Manuel Briand

Jared P Mcphillen

Mingyang Song

Nathan Crowe

Chunk based video encoding involves dividing a video into chunks with varying lengths based on the content within those frames. In contrast, dividing the video at a fix interval is prone to generating chunks starting at the middle of hard to encode areas, which can lead to a loss of encoder rate-control efficiency and produce visual quality gaps at the beginning of such chunks. The system can identify a set of boundaries for dividing the video into chunks having similar lengths and with little to no impact on visual quality. The boundaries of the chunks are placed at locations (or frames) that are far from the complex (or hard to encode) areas of the video. To do so, the system evaluates the video using various complexity metrics to identify the complex areas that require more bits to encode relative to less complex areas.

Content adaptive boundary placement for distributed encodes

In some embodiments, a method sends information for a sample of content, a first question, and a second question for output on an interface. The first question receives, from a subject, a first response for a sample level rating for an artifact that is perceived to be visible in the sample and the second question receives, from the subject, a second response for regions in the sample that are perceived to contain the artifact. The method receives the first response for the sample level rating and the second response for regions that are perceived to contain the artifact. First responses are combined from multiple subjects to generate an opinion score for the sample and second responses are combined to generate region scores for regions. The method generates training data from the opinion score and the region scores to train a process to perform an action based on the artifacts.

Subjective quality assessment tool for image/video artifacts

One embodiment of the present invention sets forth a technique for inserting content into a media program. The technique includes determining a plurality of markers corresponding to a plurality of locations within a media program. The technique also includes for each marker included in the plurality of markers, automatically analyzing a first set of intervals within the media program that lead up to the marker and a second set of intervals within the media program that immediately follow the marker and determine a set of audio characteristics associated with the first set of intervals and the second set of intervals. The technique further includes generating a plurality of scores for the plurality of markers based on the set of audio characteristics for each marker and inserting additional content at one or more markers included in the plurality of markers based on the plurality of scores.

Optimizing insertion points for content based on audio and video characteristics

A system includes a machine learning (ML) model-based video encoder configured to receive an uncompressed video sequence including multiple video frames, determine, from among the multiple video frames, a first video frame subset and a second video frame subset, encode the first video frame subset to produce a first compressed video frame subset, and identify a first decompression data for the first compressed video frame subset. The ML model-based video encoder is further configured to encode the second video frame subset to produce a second compressed video frame subset, and identify a second decompression data for the second compressed video frame subset. The first decompression data is specific to decoding the first compressed video frame subset but not the second compressed video frame subset, and the second decompression data is specific to decoding the second compressed video frame subset but not the first compressed video frame subset.

Microdosing for low bitrate video compression

In some embodiments, a method trains a first parameter of a differentiable proxy codec to encode source content based on a first loss between first compressed source content and second compressed source content that is output by a target codec. A pre-processor pre-processes a source image to output a pre-processed source image, the pre-processing being based on a second parameter. The differentiable proxy codec encodes the pre-processed source image into a compressed pre-processed source image based on the first parameter. The method determines a second loss between the source image and the compressed pre-processed source image and determines an adjustment to the first parameter based on the second loss. The adjustment is used to adjust the second parameter of the pre-processor based on the second loss.

Performance optimization of pre-processor using video proxy codec

A system includes a machine learning (ML) model-based video downsampler configured to receive an input video sequence having a first display resolution, and to map the input video sequence to a lower resolution video sequence having a second display resolution lower than the first display resolution. The system also includes a neural network-based (NN-based) proxy video codec configured to transform the lower resolution video sequence into a decoded proxy bitstream. In addition, the system includes an upsampler configured to produce an output video sequence using the decoded proxy bitstream.

Codec rate distortion compensating downsampler

A system includes processing hardware and a memory storing software code. The processing hardware executes the software code to receive automation data for media content having a default playback experience, analyze, using the automation data, at least one parameter of the media content, and generate, based on the analyzing, one or more automation instruction(s) for at least one portion(s) of the media content. The automation instruction(s) include at least one of: one or more bounding timestamps of the media content portion(s), an increased or reduced playback speed for the media content portion(s) relative to the default playback experience, or a variable playback speed for the media content portion(s). The software code is further executed to outputs the automation instruction(s) to a media delivery platform configured to distribute and control the quality of the media content or to a media player configured to automate playback of the media content.

Automation of media content playback

In some embodiments, a method generates a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video. Also, the method generates a second representation of a second relationship between bitrate and quality based on second features of a second portion of a video. The first representation is analyzed to determine a first list of bitrates for the first portion of video and the second representation is analyzed to determine a second list of bitrates for the second portion of video. The first list of bitrates is different from the second list of bitrates. The method outputs the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video.

Dynamic selection of candidate bitrates for video encoding

A system processing hardware executes a machine learning (ML) model-based video compression encoder to receive uncompressed video content and corresponding motion compensated video content, compare the uncompressed and motion compensated video content to identify an image space residual, transform the image space residual to a latent space representation of the uncompressed video content, and transform, using a trained image compression ML model, the motion compensated video content to a latent space representation of the motion compensated video content. The ML model-based video compression encoder further encodes the latent space representation of the image space residual to produce an encoded latent residual, encodes, using the trained image compression ML model, the latent space representation of the motion compensated video content to produce an encoded latent video content, and generates, using the encoded latent residual and the encoded latent video content, a compressed video content corresponding to the uncompressed video content.

Machine learning model-based video compression

According to one implementation, an image compression system includes a computing platform having a hardware processor and a system memory storing a software code. The hardware processor executes the software code to receive an input image, transform the input image to a latent space representation of the input image, and quantize the latent space representation of the input image to produce multiple quantized latents. The hardware processor further executes the software code to encode the quantized latents using a probability density function of the latent space representation of the input image, to generate a bitstream, and convert the bitstream into an output image corresponding to the input image. The probability density function of the latent space representation of the input image is obtained based on a normalizing flow mapping of one of the input image or the latent space representation of the input image.

Image compression using normalizing flows

Embodiments provide for improved stream generation. A first encoded segment is generated by encoding a first segment, of a plurality of segments in a media asset, using a first bitrate of a plurality of bitrates specified in an encoding ladder. A second encoded segment is generated by encoding the first segment using a second bitrate, where the second bitrate is lower than the first bitrate. Upon receiving a request for the first segment at the first bitrate, the second encoded segment is output based at least in part on determining that a first quality of the second encoded segment is within a tolerance of a second quality of the first encoded segment.

Segment quality-guided adaptive stream creation

Embodiments provide for improved stream generation. A target average bitrate (TAB) segment is generated by encoding a first segment, of a plurality of segments in a video, using a first maximum average bitrate (MAB) of a plurality of MABs specified in an encoding ladder. An intermediate average bitrate (IAB) segment is generated by encoding the first segment using a first intermediate bitrate, wherein the first intermediate bitrate is lower than the first MAB. Upon receiving a request for the first segment at the first MAB, the IAB segment is output based at least in part on determining that a first quality score of the IAB segment is within a predefined tolerance of a second quality score of the TAB segment.

Growing community of inventors

Kensington, CA, United States of America

Yuanyi Xue