Disney Enterprises, Inc.

Cisco Technology, Inc.

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Beijing Yojaja Software Technology Development Co., Ltd.

Eth Zurich

Inlet Technologies, Inc.

Triton US Vp Acquisition Co.

Scott C Labrozzi

Yuanyi Xue

Christopher Richard Schroers

Abdelaziz Djelouah

Erika Varis Doggett

Jared McPhillen

Michael J Bracco

Roberto Gerson De Albuquerque Azevedo

Wenhao Zhang

Carol Etta Iturralde

Eric Colin Friedrich

Leonhard Markus Helminger

Matthew Francis Caulfield

Arturo A Rodriguez

William B May, Jr

Kevin A Murray

Romann Matthew Weber

Mahesh Viveganandhan

Philip G Jacobsen

Mark D McBride

Joaquim Campos

Chetan K Mathur

Mauro Bamert

Ali C Begen

Warren Scott Wainner

Stephan Marcel Mandt

Xuchang Huangfu

James D Zimmerman

Yang Zhang

Kevin Scott Kirkup

Anne K McCormick

Nicholas David Gregory

David R Oran

Chen Liu

Qi Rong Wang

William C VerSteeg

Joshua B Gahm

Tunc Ozan Aydin

Xiaobo Liu

Lemei Huang

Gary Shaffer

Mitchel C Jacobs

Erika Elizabeth Varis Doggett

Katherine Hinsen

Clara Fernandez Labrador

Mingyang Song

Nathan Crowe

Joan Massich Vall

Eitan Abecassis

Cafer Mertcan Akcay

Simon Meierhans

Jared P Mcphillen

Manuel Briand

Mugdha Oltikar

Neal S Page

Adrian P Phillips

James M Hill

James Christopher Akers

Christopher Schroers

Romann Weber

Chunk based video encoding involves dividing a video into chunks with varying lengths based on the content within those frames. In contrast, dividing the video at a fix interval is prone to generating chunks starting at the middle of hard to encode areas, which can lead to a loss of encoder rate-control efficiency and produce visual quality gaps at the beginning of such chunks. The system can identify a set of boundaries for dividing the video into chunks having similar lengths and with little to no impact on visual quality. The boundaries of the chunks are placed at locations (or frames) that are far from the complex (or hard to encode) areas of the video. To do so, the system evaluates the video using various complexity metrics to identify the complex areas that require more bits to encode relative to less complex areas.

Content adaptive boundary placement for distributed encodes

In some embodiments, a method sends information for a sample of content, a first question, and a second question for output on an interface. The first question receives, from a subject, a first response for a sample level rating for an artifact that is perceived to be visible in the sample and the second question receives, from the subject, a second response for regions in the sample that are perceived to contain the artifact. The method receives the first response for the sample level rating and the second response for regions that are perceived to contain the artifact. First responses are combined from multiple subjects to generate an opinion score for the sample and second responses are combined to generate region scores for regions. The method generates training data from the opinion score and the region scores to train a process to perform an action based on the artifacts.

Subjective quality assessment tool for image/video artifacts

A system includes a hardware processor and a memory storing a video/audio (V/A) synchronizer including video and audio encoders. The hardware processor executes the V/A synchronizer to receive raw video and audio extracted from media content, partition the raw video into video frame patches, partition the raw audio into audio samples, pre-process the video frame patches and the audio samples for encoding. The hardware processor further executes the V/A synchronizer to encode, using the video encoder, the pre-processed video frame patches to provide pre-processed and encoded video frame patches used to provide a latent representation of the raw video, encode, using the audio encoder, the pre-processed audio samples to provide pre-processed and encoded audio samples used to provide a latent representation of the raw audio, and synchronize, using the latent representations of the raw video and the raw audio, the raw audio with the raw video.

Video and audio synchronization with dynamic frame and sample rates

One embodiment of the present invention sets forth a technique for inserting content into a media program. The technique includes determining a plurality of markers corresponding to a plurality of locations within a media program. The technique also includes for each marker included in the plurality of markers, automatically analyzing a first set of intervals within the media program that lead up to the marker and a second set of intervals within the media program that immediately follow the marker and determine a set of audio characteristics associated with the first set of intervals and the second set of intervals. The technique further includes generating a plurality of scores for the plurality of markers based on the set of audio characteristics for each marker and inserting additional content at one or more markers included in the plurality of markers based on the plurality of scores.

Optimizing insertion points for content based on audio and video characteristics

A system includes a machine learning (ML) model-based video encoder configured to receive an uncompressed video sequence including multiple video frames, determine, from among the multiple video frames, a first video frame subset and a second video frame subset, encode the first video frame subset to produce a first compressed video frame subset, and identify a first decompression data for the first compressed video frame subset. The ML model-based video encoder is further configured to encode the second video frame subset to produce a second compressed video frame subset, and identify a second decompression data for the second compressed video frame subset. The first decompression data is specific to decoding the first compressed video frame subset but not the second compressed video frame subset, and the second decompression data is specific to decoding the second compressed video frame subset but not the first compressed video frame subset.

Microdosing for low bitrate video compression

A system includes a computing platform having processing hardware, and a memory storing software code. The software code is executed to receive digital content indexed to a timeline, receive insertion data identifying a timecode of the timeline, and encode the digital content using the insertion data to provide segmented content having a segment boundary at the timecode, and first and second segments adjoining the segment boundary, wherein the first segment precedes, and the second segment succeeds, the segment boundary. The software code also re-processes the first and second segments to apply a fade-out within or to the first segment and a fade-in within or to the second segment, wherein re-processing the first and second segments provides encoded segments having the segment boundary configured as an insertion point for supplemental content.

Targeted re-processing of digital content

In some embodiments, a method trains a first parameter of a differentiable proxy codec to encode source content based on a first loss between first compressed source content and second compressed source content that is output by a target codec. A pre-processor pre-processes a source image to output a pre-processed source image, the pre-processing being based on a second parameter. The differentiable proxy codec encodes the pre-processed source image into a compressed pre-processed source image based on the first parameter. The method determines a second loss between the source image and the compressed pre-processed source image and determines an adjustment to the first parameter based on the second loss. The adjustment is used to adjust the second parameter of the pre-processor based on the second loss.

Performance optimization of pre-processor using video proxy codec

A system includes a machine learning (ML) model-based video downsampler configured to receive an input video sequence having a first display resolution, and to map the input video sequence to a lower resolution video sequence having a second display resolution lower than the first display resolution. The system also includes a neural network-based (NN-based) proxy video codec configured to transform the lower resolution video sequence into a decoded proxy bitstream. In addition, the system includes an upsampler configured to produce an output video sequence using the decoded proxy bitstream.

Codec rate distortion compensating downsampler

A system includes processing hardware and a memory storing software code. The processing hardware executes the software code to receive automation data for media content having a default playback experience, analyze, using the automation data, at least one parameter of the media content, and generate, based on the analyzing, one or more automation instruction(s) for at least one portion(s) of the media content. The automation instruction(s) include at least one of: one or more bounding timestamps of the media content portion(s), an increased or reduced playback speed for the media content portion(s) relative to the default playback experience, or a variable playback speed for the media content portion(s). The software code is further executed to outputs the automation instruction(s) to a media delivery platform configured to distribute and control the quality of the media content or to a media player configured to automate playback of the media content.

Automation of media content playback

In some embodiments, a method generates a first representation of a first relationship between bitrate and quality based on first features of a first portion of a video. Also, the method generates a second representation of a second relationship between bitrate and quality based on second features of a second portion of a video. The first representation is analyzed to determine a first list of bitrates for the first portion of video and the second representation is analyzed to determine a second list of bitrates for the second portion of video. The first list of bitrates is different from the second list of bitrates. The method outputs the first list of bitrates for use encoding the first portion of video and the second list of bitrates for use encoding the second portion of video.

Dynamic selection of candidate bitrates for video encoding

A system includes a computing platform having processing hardware, and a memory storing software code. The software code is executed to receive content having a sequence of content segments, and marker data identifying a location within the sequence, identify, using the content and the marker data, segment boundaries of a content segment containing the location, determine, using the location and the segment boundaries, whether the location is situated within a predetermined interval of one of the segment boundaries, and re-encode a subsection of the sequence to produce a new segment boundary at the location. When the location is not situated within the predetermined interval, the subsection of the sequence includes the content segment containing the location. When the location is situated within the predetermined interval, the subsection of the sequence includes the content segment containing the location and a content segment adjoining the content segment containing the location.

Surgical micro-encoding of content

A system processing hardware executes a machine learning (ML) model-based video compression encoder to receive uncompressed video content and corresponding motion compensated video content, compare the uncompressed and motion compensated video content to identify an image space residual, transform the image space residual to a latent space representation of the uncompressed video content, and transform, using a trained image compression ML model, the motion compensated video content to a latent space representation of the motion compensated video content. The ML model-based video compression encoder further encodes the latent space representation of the image space residual to produce an encoded latent residual, encodes, using the trained image compression ML model, the latent space representation of the motion compensated video content to produce an encoded latent video content, and generates, using the encoded latent residual and the encoded latent video content, a compressed video content corresponding to the uncompressed video content.

Machine learning model-based video compression

According to one implementation, an image compression system includes a computing platform having a hardware processor and a system memory storing a software code. The hardware processor executes the software code to receive an input image, transform the input image to a latent space representation of the input image, and quantize the latent space representation of the input image to produce multiple quantized latents. The hardware processor further executes the software code to encode the quantized latents using a probability density function of the latent space representation of the input image, to generate a bitstream, and convert the bitstream into an output image corresponding to the input image. The probability density function of the latent space representation of the input image is obtained based on a normalizing flow mapping of one of the input image or the latent space representation of the input image.

Image compression using normalizing flows

Growing community of inventors

Cary, NC, United States of America

Scott C Labrozzi