Microsoft Technology Licensing, LLC

Microsoft Corporation

Stacey Spears

Thomas W Holcomb

Chuang Gu

Chih-Lung Lin

Glenn Frederick Evans

Donald J Munsil

Chun-Wei Chan

William Chen

Yaming He

Haoyun Wu

Florin Folta

Ming-Chieh Lee

Cheng Chang

Wenbo Zhang

Sudhakar Visweswara Prabhu

King Wei Hor

Xiping Zuo

Philip Lawrence Garrett

Amy Jennifer Beauford

Anil Murching

Shijun Sun

Sanjeev Mehrotra

Bruce W Randall

Matt Dubois

Minesh G Shilotri

An adaptive video noise reduction (AVNR) engine reduces noise in streaming video. The engine reads embedded information in the streaming video describing attributes of the video, such as bit rate, frame rate, resolution, content type and quality of the streaming video. The information may be included in a manifest of the streaming video. Based on the video attributes, the engine outputs one or more input values to applications that reduce noise of the streaming video. The one or more applications may reduce mosquito, block, temporal, random and banding noise of the video in response to the input values from the engine.

Adaptive noise reduction engine for streaming video

In some examples, techniques and architectures for modifying resolution of video displayed on a display device include receiving information from a distance sensor and one or more image sensors, detecting an activity of a viewer of the display device based, at least in part, on the information received from the distance sensor and the one or more image sensors, and transmitting a request to a remote server to adjust bandwidth of media content streaming to the display device to a new bandwidth. The new bandwidth may be based, at least in part, on one or more of the activity of the viewer, size of the display device, or an ability of the viewer to resolve images displayed on the display device.

Dynamic video streaming based on viewer activity

Systems and methods for evaluating video quality of HDMI data are provided. In one example an HDMI video quality evaluator controls a video device to render a test pattern and encode the pattern into HDMI. The video device is controlled to transmit the HDMI formatted pattern from an output port to an input port on the device, capture a frame of the pattern, and transmit the frame to the quality evaluator. The quality evaluator compares rendered pixels in the frame to test pixels in the test pattern to identify video quality errors. Errors that exceed a threshold are identified, and a test result is outputted that includes errors exceeding the threshold.

HDMI image quality analysis

Techniques and tools are presented for controlling artifacts such as banding artifacts, ringing artifacts and film scan artifacts in video. For example, before encoding, a pre-processor performs combined filtering and dithering on video such that the weight of dithering at a location depends on the results of filtering at the location. For the combined filtering and dithering, the pre-processor can determine a lowpass signal and highpass residual, weight dithering based on local characteristics of the highpass residual, and then combine the lowpass signal with the weighted dithering. Or, to determine the relative weight, the pre-processor can use a filter whose normalization factor varies depending on how many sample values around a location are within a threshold of similarity to a current sample value at the location. The filtering and dithering can use different strengths for luma and chroma channels.

Filtering and dithering as pre-processing before encoding

A multiple bitrate (MBR) video encoding management tool utilizes available processing units for parallel MBR video encoding. Instead of focusing only on multi-threading of encoding tasks for a single picture or group of pictures (GOP), the management tool parallelizes the encoding of multiple GOPs between different processing units and/or different computing systems. With this parallel MBR video encoding architecture, different GOPs can be encoded in parallel. To facilitate such parallel encoding, data dependencies between GOPs are removed. The management tool can adjust the number of GOPs to encode in parallel on a computing system so as to favor parallelism of encoding for different GOPs at the expense of parallelism of encoding inside a GOP, or vice versa, and thereby set a suitable balance between encoding latency and throughput.

Parallel multiple bitrate video encoding to reduce latency and dependences between groups of pictures

Quality settings established by an encoder are adjusted based on information associated with regions of interest ('ROIs'). For example, quantization step sizes can be reduced (to improve quality) or increased (to reduce bit rate). ROIs can be identified and quality settings can be adjusted based on input received from a user interface. An overlap setting can be determined for a portion of a picture that corresponds to an ROI overlap area. For example, an overlap setting is chosen from step sizes corresponding to a first overlapping ROI and a second overlapping ROI, or from relative reductions in step size corresponding to the first ROI and the second ROI. ROIs can be parameterized by information (e.g., using data structures) that indicates spatial dimensions of the ROIs and quality adjustment information (e.g., dead zone information, step size information, and quantization mode information).

Regions of interest for quality adjustments

Strategies for effectively discovering, selecting, configuring, and controlling components used in media processing applications are described. According to one exemplary implementation, the strategies described configure the components based on profile information, configuration information, and a hierarchical ordering of configuration parameters. The hierarchical ordering may combine different coding paradigms, where one or more high level nodes in the ordering may define configuration parameters which are common to multiple coding paradigms. In this ordering, selection of a configuration parameter may cascade down to affect lower-ranking dependent parameters in the hierarchical ordering. According to one advantage, the hierarchical ordering provides a more uniform, extensible, and problem-free approach to configuring components than unstructured approaches to configuration. Moreover, applications can utilize the hierarchical ordering at different levels of granularity.

Strategies for configuring media processing functionality using a hierarchical ordering of control parameters

The subject disclosure relates to face recognition in video. Face detection data in frames of input data are used to generate face galleries, which are labeled and used in recognizing faces throughout the video. Metadata that associates the video frame and the face are generated and maintained for subsequent identification. Faces other than those found by face detection may be found by face tracking, in which facial landmarks found by the face detection are used to track a face over previous and/or subsequent video frames. Once generated, the maintained metadata may be accessed to efficiently determine the identity of a person corresponding to a viewer-selected face.

Face recognition in video content

A video encoding system encodes video streams for multiple bit rate video streaming using an approach that permits the encoded bit rate to vary subject to a peak bit rate and average bit rate constraints for higher quality streams, while a bottom bit rate stream is encoded to achieve a constant chunk rate. The video encoding system also dynamically decides an encoding resolution for segments of the multiple bit rate video streams that varies with video complexity so as to achieve a better visual experience for multiple bit rate streaming.

Multiple bit rate video encoding using variable bit rate and dynamic resolution for adaptive video streaming

A video encoder uses previously calculated motion information for inter frame coding to achieve faster computation speed for video compression. In a multi bit rate application, motion information produced by motion estimation for inter frame coding of a compressed video bit stream at one bit rate is passed on to a subsequent encoding of the video at a lower bit rate. The video encoder chooses to use the previously calculated motion information for inter frame coding at the lower bit rate if the video resolution is unchanged. A multi core motion information pre-calculation produces motion information prior to encoding by dividing motion estimation of each inter frame to separate CPU cores.

Video encoding using previously calculated motion information

A video encoding system encodes video streams for multiple bit rate video streaming using an approach that permits the encoded resolution to vary based, at least in part, on motion complexity. The video encoding system dynamically decides an encoding resolution for segments of the multiple bit rate video streams that varies with video complexity so as to achieve a better visual experience for multiple bit rate streaming. Motion complexity may be considered separately, or along with spatial complexity, in making the resolution decision.

Motion based dynamic resolution multiple bit rate video encoding

Techniques and tools for content classification and adaptive quantization are described. In an example implementation, a video encoding tool classifies blocks as textured, dark smooth or other smooth. The tool classifies a block as textured or non-textured by comparing the energy of AC coefficients for the block to a texture threshold, which can be set using a non-linear mapping of possible texture classification levels to possible texture thresholds. If a block is not textured, the tool further classifies the block as dark smooth or smooth depending on average intensity value for the block. Using the classification information and one or more control parameters to control bit allocation for dark smooth content relative to other smooth content, the tool encodes the video and outputs encoded video information. Example multi-pass approaches to setting the control parameters are also described.

Classifying and controlling encoding quality for textured, dark smooth and smooth video content

Whether interlaced video fields form a progressive video frame can be automatically determined. The presence or absence of a first characteristic of one or more video fields can be determined by analysis of the fields and/or related information such as flags, cadence, previous determinations, and others. Similarly, the presence or absence of a second characteristic can be detected. In accordance with the detecting, how likely or whether the two or more video fields form a progressive video frame can be determined based on a possibly predetermined likelihood that fields of progressive video frames in general have or do not have the first characteristic and based on a possibly predetermined likelihood that fields of interlaced video frames in general have or do not have the second characteristic.

Growing community of inventors

Sammamish, WA, United States of America

Stacey Spears