Microsoft Technology Licensing, LLC

Walid Ali

Ce Wang

Rony Ferzli

Regis J Crinon

Bharath Siravara

Kandarp Jani

Kurt David Wrisley

Shiwei Wang

Stephen Clifford Cooper

Kevin John Smith

Tolga Kilicli

Robert A Farrow

Bruce Lin

Shankar Lakshmi Regunathan

Guo-Wei Shieh

Ronald Boskovic

Peter Zuo

Nelson Philip Pinto

Embodiments are configured to provide interactive communication functionality including adaptive video processing functionality that can be used to process aspects of a video signal, but the embodiments are not so limited. In an embodiment, components of a video conferencing system can operate to provide a video signal based in part on the use of adaptive processing features which include scaling and/or other pixel processing features. In one embodiment, components of an interactive video system can operate to adaptively manage and control video payload parameters to adapt to various communication conditions associated with a real-time or near-real time interactive video environment.

Adaptive video processing of an interactive environment

A device may generate a media stream to be shared with other users by building a media graph, comprising a series of interconnected processing units that perform various processing tasks. However, the time involved in generating the media graph may delay the initialization of the media stream, and adjusting properties of the media stream (such as resolution or codec) may result in an interruption of the media stream while a new media graph is built. Instead, a media graph cache may be provided to cache a set of media graphs, which may be interchangeably selected for rapid initialization and adjusting of media stream properties. The media component (e.g., a videocamera) may also be configured to promote rapid adjustments to some media stream properties, while maintaining other properties (e.g., field of view and white balance) for a smooth transition between media stream property sets.

Rapidly initializing and dynamically adjusting media streams

Architecture for enhancing the compression (e.g., luma, chroma) of a video signal and improving the perceptual quality of the video compression schemes. The architecture operates to reshape the normal multimodal energy distribution of the input video signal to a new energy distribution. In the context of luma, the algorithm maps the black and white (or contrast) information of a picture to a new energy distribution. For example, the contrast can be enhanced in the middle range of the luma spectrum, thereby improving the contrast between a light foreground object and a dark background. At the same time, the algorithm reduces the bit-rate requirements at a particular quantization step size. The algorithm can be utilized also in post-processing to improve the quality of decoded video.

Entropy coding efficiency enhancement utilizing energy distribution remapping

Techniques to perform fast motion estimation are described. An apparatus may comprise a motion estimator operative to receive as input a current frame and a reference frame from a digital video sequence. The motion estimator may generate and output a motion vector. The motion vector may represent a change in position between a current block of the current frame and a matching reference block of the reference frame. The motion estimator may utilize an enhanced block matching technique to perform block matching based on stationary and spatially proximate blocks. Other embodiments are described and claimed.

Techniques to perform fast motion estimation

Technologies for recovering from dropped frames in the real-time transmission of video over an IP network are provided. A video streaming module receives a notification from a receiving module that a data packet has been lost. The video streaming module determines, based on the type of video frame conveyed in the lost packet and the timing of the lost packet in relation to the sequence of video frames transmitted to the receiving module, whether or not a replacement video frame should be sent to the receiving module. If the video streaming module determines a replacement video frame is warranted, then the video streaming module instructs a video encoding module to generate a replacement video frame and then transmits the replacement video frame to the receiving module.

Recovering from dropped frames in real-time transmission of video over IP networks

A device may generate a media stream to be shared with other users by building a media graph, comprising a series of interconnected processing units that perform various processing tasks. However, the time involved in generating the media graph may delay the initialization of the media stream, and adjusting properties of the media stream (such as resolution or codec) may result in an interruption of the media stream while a new media graph is built. Instead, a media graph cache may be provided to cache a set of media graphs, which may be interchangeably selected for rapid initialization and adjusting of media stream properties. The media component (e.g., a video camera) may also be configured to promote rapid adjustments to some media stream properties, while maintaining other properties (e.g., field of view and white balance) for a smooth transition between media stream property sets.

A perceptual mechanism for residue selection in a video encoder may be provided. The mechanism may comprise a method, system, or device for receiving video frames comprising pluralities of pixels. For each video frame, a sensitivity threshold may be determined for each pixel of a previous video frame. The pixels of the video frame may compared in turn to the pixels of the previous video frame to determine a residue value. The residue value may be compared to the sensitivity threshold such that when the residue value is less than the sensitivity threshold, the pixel data in the video frame may be zeroed out prior to encoding the video frame for transmission.

Perceptual mechanism for the selection of residues in video coders

Architecture that employs texture sensitive temporal filtering to reuse motion estimation information in a realtime encoder. The temporal filter is applied for classified static areas. The architecture reuses the motion estimation results on motion vectors, cost estimates (e.g., sum of absolute difference (SAD)), and edge awareness texture information to apply the temporal filter on the current picture. Filtering can be applied at the pixel level, block level or macroblock level.

Texture sensitive temporal filter based on motion estimation

Architecture for accelerating video compression by using the motion vectors produced locally by a camera is disclosed. Video frames are captured by the camera (e.g., a webcam) which also computes a motion vector for the frames. Metadata can also be generated that represent an index of motion quality associated with the motion vector. The motion vector is passed to a video compression engine which selectively uses the motion vector directly or alternatively as a seed for a compression and encoding algorithm. This algorithm produces a compressed video frame representing a motion estimate having a selected motion quality index value. In this way, complexity is reduced in the video compression engine, resulting in faster and more efficient video compression. Alternatively, the webcam sends a compressed video bitstream to reduce throughput on the connection and the receiving computing system processes residual information to derive an estimate of the quality index for each macroblock/kernel.

Digital video compression acceleration based on motion vectors produced by cameras

Embodiments are configured to provide video conferencing functionality including using region of interest (ROI) features to provide a video signal, but the embodiments are not so limited. In an embodiment, components of a video conferencing system can operate to provide a video signal using pixel data associated with a ROI. In one embodiment, a video conference device can include a detector that can be used to detect human flesh tone regions in a video scene as part of providing a video stream to one or more conference participants.

Video region of interest features

Real-time video processing functionality may be provided using pre-processing and/or post-processing features to provide a video signal. Components of a real-time video processing system may operate to receive a real-time video signal. The real-time video signal may be downscaled based in part on the use of features of a pre-processing component applying a downscale polyphase filter that may be used to compensate for bandwidth constraints associated with a real-time video conferencing environment. The downscaled real-time video may be communicated across a network, such as the Internet. Upon receipt of the downscaled real-time video, the downscaled real-time video may be upscaled based in part on the use of features of a post-processing component applying an upscale polyphase filter.

Processing real-time video

Embodiments are configured to provide video conferencing functionality including using pre-processing and/or post-processing features to provide a video signal, but the embodiments are not so limited. In an embodiment, components of a video conferencing system can operate to provide a video signal based in part on the use of features of a pre-processing component and/or post-processing component. In one embodiment, a video conference device can include a pre-processing component and/or post-processing component to that can be used to compensate for bandwidth constraints associated with a video conferencing environment.

Growing community of inventors

Sammamish, WA, United States of America

Walid Ali