Netflix Inc.

Meta Platforms, Inc.

Facebook, Inc.

Fosco Bianchetti

Ioannis Katsavounidis

Mark Watson

Anne Aaron

Jan De Cock

Aaron Hipple

Will Wolcott

David Randall Ronca

Aditya Mavlankar

Ping-Hao Wu

Liwei Guo

Steven Kang

Rama Kalluri

Gaurang Chaudhari

Andy Schuler

Maxim Bykov

Volodymyr Kondratenko

David Ronca

In various embodiments, a buffer-based encoding application generates a first convex hull of subsequence encode points based on multiple encoding points and a first subsequence associated with a media title. The buffer-based encoding application then generates a first global convex hull of media encode points based on a transmission buffer constraint, the first convex hull, and a second global convex hull of media encode points. Notably, the second global convex hull is associated with a portion of the media title that occurs before the first subsequence in a playback order for the media title. Subsequently, the subsequence-based encoding application selects a first media encode point included in the first global convex hull based on a media metric and determines a first encoded media sequence based on the selected media encode point. The first encoded media sequence is subsequently streamed to an endpoint device during playback of the media title.

Optimizing encoding operations when generating a buffer-constrained version of a media title

A method includes receiving, with a computing system, data representing a video item into a buffer. The method further includes outputting the video item from the buffer to a display system. The method further includes determining that utilization of the buffer falls below a predetermined threshold. The method further includes, in response to determining that the utilization of the buffer falls below the predetermined threshold, determining that there is a specified rebuffering point within a predetermined time frame. The method further includes pausing with the computing system, the video item at the specified rebuffering point in response to determining that there is the specified rebuffering point within the predetermined time frame.

Temporal placement of a rebuffering event

One embodiment of the invention sets forth a mechanism for encoding video streams associated with the same digital content such that switch points staggered across two video streams occur at every offset temporal distance. The offset temporal distance is less than the distance between two consecutive key frames in a given video stream. This enables a content player to switch to a video stream having a playback quality up or down one level from a current video stream at the offset temporal distance from the most recently played key frame. In effect, the content player does not wait the entire key frame temporal distance before switching.

Staggered key frame video encoding

In various embodiments, an encoding optimization application encodes a source video sequence. The encoding optimization application partitions a source video sequence associated with a media title into multiple subsequences that include a first subsequence and a second subsequence. The encoding optimization application then generates a first encoded subsequence based on the first subsequence and a first resolution. The encoding optimization application generates a second encoded subsequence based on the second subsequence and a second resolution. Notably, the second resolution is higher than the first resolution. The encoding optimization application then aggregates the first encoded subsequence and the second encoded subsequence to generate an encoded video sequence. At least a portion of the encoded video sequence is subsequently streamed to an endpoint device during a playback of the media title.

Techniques for selecting resolutions for encoding different shot sequences

A shot analyzer varies the resolution when generating encoded video sequences for streaming. The shot analyzer generates a first encoded video sequence based on a first resolution and a source video sequence that is associated with a video title. The shot analyzer then determines a first encoded shot sequence from multiple encoded shot sequences included in the first encoded video sequence based on quality metric(s). The first encoded shot sequence is associated with a first shot sequence included in the source video sequence. Subsequently, the shot analyzer generates a second encoded shot sequence based on a second resolution and the first shot sequence. The shot analyzer generates a second encoded video sequence based on the first encoded video sequence and the second encoded shot sequence. At least a first portion of the second encoded video sequence is subsequently streamed to an endpoint device during playback of the video title.

Encoding techniques for optimizing distortion and bitrate

In various embodiments, a sequence-based encoding application partitions a set of shot sequences associated with a media title into multiple clusters based on at least one feature that characterizes media content and/or encoded media content associated with the media title. The clusters include at least a first cluster and a second cluster. The sequence-based encoding application encodes a first shot sequence using a first operating point to generate a first encoded shot sequence. The first shot sequence and the first operating point are associated with the first cluster. By contrast, the sequence-based encoding application encodes a second shot sequence using a second operating point to generate a second encoded shot sequence. The second shot sequence and the second operating point are associated with the second cluster. Subsequently, the sequence-based encoding application generates an encoded media sequence based on the first encoded shot sequence and the second encoded shot sequence.

Optimizing encoding operations when generating encoded versions of a media title

An encoding engine encodes a video sequence to provide optimal quality for a given bitrate. The encoding engine cuts the video sequence into a collection of shot sequences. Each shot sequence includes video frames captured from a particular capture point. The encoding engine resamples each shot sequence across a range of different resolutions, encodes each resampled sequence with a range of quality parameters, and then upsamples each encoded sequence to the original resolution of the video sequence. For each upsampled sequence, the encoding engine computes a quality metric and generates a data point that includes the quality metric and the resample resolution. The encoding engine collects all such data points and then computes the convex hull of the resultant data set. Based on all convex hulls across all shot sequences, the encoding engine determines an optimal collection of shot sequences for a range of bitrates.

Encoding technique for optimizing distortion and bitrate

In various embodiments, an encoder comparison application compares the performance of different configured encoders. In operation, the encoder comparison application generates a first global convex hull of video encode points based on a first configured encoder and a set of subsequences included in a source video sequence. Each video encode point is associated with a different encoded version of the source video sequence. The encoder comparison application also generates a second global convex hull of video encode points based on a second configured encoder and the subsequences. Subsequently, the encoder configuration application computes a performance value for an encoding comparison metric based on the first global convex hull and the second global convex hull. Notably, the first performance value estimates a difference in performance between the first configured encoder and the second configured encoder.

Comparing video encoders/decoders using shot-based encoding and a perceptual visual quality metric

In various embodiments, an iterative encoding application encodes a source video sequence. The encoding optimization application generates a set of shot encode points based on a set of encoding points and a first shot sequence included in the source video sequence. Each shot encode point is associated with a different encoded shot sequence. The encoding optimization application performs convex hull operation(s) across the set of shot encode points to generate a first convex hull associated with the first shot sequence. Subsequently, the encoding optimization application generates encoded video sequences based on the first convex hull and a second convex hull associated with a second shot sequence included in the source video sequence. The encoding optimization application computes a new encoding point based on the encoded video sequences and a target value for a first video metric and then generates an optimized encoded video sequence based on the new encoding point.

Iterative techniques for encoding video content

In various embodiments, an encoding optimization application positions key frames within encoded video sequences based on shot changes. The encoding optimization application determines key frame location(s) based on shot change(s) included in a source video sequence associated with a media title. Each key frame location is associated with a different frame included in the source video sequence. For each of the key frame location(s), the encoding optimization application configures an encoding application to encode a frame of video content located at the key frame location as a key frame when performing encoding operations. Subsequently, the encoding optimization application causes the encoding application to perform encoding operation(s) on the source video sequence to generate a first encoded video sequence. During playback, the media title is switchable between a decoded version of the first encoded video sequence and a decoded version of a second encoded video sequence at the key frame location(s).

Techniques for positioning key frames within encoded video sequences

The disclosed computer-implemented method may include, for a current frame of a sequence of video frames, determining a frame type label of the current frame. The method may include, in response to determining that the current frame is labeled as an intra frame (I-frame), decoding the current frame and comparing the decoded frame to historical I-frame data. The method may also include, in response to the comparison satisfying a shot-change threshold, flagging the current frame as a shot-change frame, and in response to flagging the current frame as the shot-change frame, storing the current frame for a subsequent shot-change detection. The method may further include updating, based on flagged shot-change frames, shot boundaries for the sequence of video frames. Various other methods, systems, and computer-readable media are also disclosed.

Shot-change detection using container level information

In various embodiments, a subsequence-based encoding application generates subsequences based on a source sequence associated with a media title. The subsequence-based encoding application then encodes both a first subsequence and a second subsequence across each of multiple configured encoders and at least one rate control value to generate, respectively, a first set of encoded subsequences and a second set of encoded subsequences. Notably, each configured encoder is associated with a combination of an encoder and a configuration, and at least two configured encoders are different from one another. Subsequently, the subsequence-based encoding application generates encoded media sequences based on the first set of encoded subsequences and the second set of encoded subsequences. Finally, the application selects a first encoded media sequence from the encoded media sequences based on a first target value for a media metric to subsequently stream to a first endpoint device during playback of the media title.

Growing community of inventors

San Jose, CA, United States of America

Ioannis Katsavounidis