Microsoft Technology Licensing, LLC

Facebook, Inc.

Meta Platforms, Inc.

Microsoft Corporation

Yaming He

Zhengping Zuo

Chuang Gu

Weidong Zhao

Stacey Spears

Florin Folta

King Wei Hor

Chun-Wei Chan

Shankar Lakshmi Regunathan

Amit Puntambekar

William Chen

Michael Hamilton Coward

Minchuan Chen

Sonal Gandhi

Minesh G Shilotri

Embodiments are described for dynamically adapting video encoding to maintain a nearly stable frame rate based on processor capabilities and bandwidth, for example, by varying a quantization parameter. The quality of the encoded video can be varied to maintain the nearly constant frame rate, which may be measured from the number of encoded video frames being transmitted over a network interface.

Dynamic codec adaptation

A video encoder may be put in a starve mode during a low latency operation in which the video encoder may be operated in a mode that allows video frames to be encoded without any interdependencies such as motion compensation. At least one encoding parameter of the video encoding is selected such that, for each resulting encoded video frame, the video frame fits in exactly one application layer packet.

Video encoding using starve mode

In one embodiment, a method includes receiving multimedia content information associated with at least one segment of a multimedia content, receiving a request to view the at least one segment of the multimedia content from a client device, logging playback information associated with the viewing of the at least one segment of the multimedia content, determining a multimedia quality metric associated with the at least one segment of multimedia content based in part upon a portion of the received multimedia content information and a portion of the logged playback information, and classifying the at least one segment of the multimedia content with the multimedia quality metric.

Visual quality metrics

Dynamic codec adaption

Decoding tasks are identified for decoding encoded video. Decoding tasks may include entropy decoding tasks, motion compensation tasks, inverse frequency transform tasks, inverse quantization tasks, intra decoding tasks, loop filtering tasks, or other tasks. Task dependencies are identified for the video decoding tasks. For example, one or more decoding tasks may depend on prior completion of entropy decoding tasks. The decoding tasks are prioritized based at least in part on the task dependencies. For example, a higher priority may be assigned to tasks that must be completed before other tasks that depend on them can begin. Prioritized decoding tasks are selected to be performed by hardware threads. For example, a first hardware thread may perform a first decoding task that does not depend on any uncompleted tasks while a second hardware thread performs a second decoding task that does not depend on any uncompleted tasks.

Scalable multi-thread video decoding

A multiple bitrate (MBR) video encoding management tool utilizes available processing units for parallel MBR video encoding. Instead of focusing only on multi-threading of encoding tasks for a single picture or group of pictures (GOP), the management tool parallelizes the encoding of multiple GOPs between different processing units and/or different computing systems. With this parallel MBR video encoding architecture, different GOPs can be encoded in parallel. To facilitate such parallel encoding, data dependencies between GOPs are removed. The management tool can adjust the number of GOPs to encode in parallel on a computing system so as to favor parallelism of encoding for different GOPs at the expense of parallelism of encoding inside a GOP, or vice versa, and thereby set a suitable balance between encoding latency and throughput.

Parallel multiple bitrate video encoding to reduce latency and dependences between groups of pictures

Various examples are disclosed herein that relate to staged element classification. For example, one disclosed example provides a method of classifying elements by forming elements for classification into a plurality of first-level sets in a first stage, generating primary groups within the first-level sets based on element similarity, forming a plurality of second-level sets from the first-level sets in a second stage, generating secondary groups within the second-level sets based on element similarity, and merging a plurality of the primary and/or secondary groups based on element similarity.

Staged element classification

The subject disclosure relates to face recognition in video. Face detection data in frames of input data are used to generate face galleries, which are labeled and used in recognizing faces throughout the video. Metadata that associates the video frame and the face are generated and maintained for subsequent identification. Faces other than those found by face detection may be found by face tracking, in which facial landmarks found by the face detection are used to track a face over previous and/or subsequent video frames. Once generated, the maintained metadata may be accessed to efficiently determine the identity of a person corresponding to a viewer-selected face.

Face recognition in video content

A video encoding system encodes video streams for multiple bit rate video streaming using an approach that permits the encoded resolution to vary based, at least in part, on motion complexity. The video encoding system dynamically decides an encoding resolution for segments of the multiple bit rate video streams that varies with video complexity so as to achieve a better visual experience for multiple bit rate streaming. Motion complexity may be considered separately, or along with spatial complexity, in making the resolution decision.

Growing community of inventors

Redmond, WA, United States of America

Yaming He