Microsoft Technology Licensing, LLC

Qformx, Inc.

Calista Technologies, Inc.

Calister Technologies

Zhigu Holdings Limited

B Anil Kumar

Lihua Zhu

Sridhar Sankuratri

Shir Aharon

Jeroen E Van Eesteren

Gary Joseph Sullivan

Nadim Y Abdo

Olof L E Mases

Feng Wu

Krishna Mohan Malladi

Yongjun Wu

Shyam Sadhwani

Hao Zhang

Parag Chakraborty

Bradley Post

Meher P Malakapalli

Guosheng Sun

Jizheng Xu

Sandeep Kanumuri

Neal David Margulis

Costin Hagiu

Vladimir Stoyanov

Asael Dror

Karthik Lakshminarayanan

Vladimir Pavlov

Victor Ramamoorthy

Paul Lessard

Nelamangal Krishnaswamy Srinivas

Joy Chik

Wilhelm R Schmieder

Mahesh Sharad Lotlikar

Tad Dennis Brockway

Tong L Wynn

Damien Gerard Saint Macary

Hon Luu

Jayashree Sadagopan

Eric K Han

Lin Tan

Artem Belkine

Paul Johan Hsieh

Charles Ogden

Jeff B Widergren

Winston Matthew Penfold Johnston

Eric Kai-hau Han

Winston M P Johnston

Alp Emre Kanlikilicer

Ricardo A Baratto

Michael E Seydl

Meetesh Barua

Pandele Stanescu

Jeroen Dirk Meijer

Frank Palita

Eric Rall

Winston M Johnston

Jingyaw Sun

Arjun Dube

Olof LE Mases

Tad Brockway

Tong Wynn

Meher Prasad Malakapalli

Innovations in the area of prediction of block vector ('By') values improve encoding or decoding of blocks using intra block copy (“BC”) prediction. For example, some of the innovations relate to use of a default BV predictor with a non-zero value. Other innovations relate to use of a selected one of multiple BV predictor candidates for a current block. Still other innovations relate to use of a skip mode in which a current intra-BC-predicted block uses a predicted BV value.

Block vector prediction in video and image coding/decoding

Innovations in the area of prediction of block vector ('BV') values improve encoding or decoding of blocks using intra block copy (“BC”) prediction. For example, some of the innovations relate to use of a default BV predictor with a non-zero value. Other innovations relate to use of a selected one of multiple BV predictor candidates for a current block. Still other innovations relate to use of a skip mode in which a current intra-BC-predicted block uses a predicted BV value.

Innovations in the area of prediction of block vector ('BV') values improve encoding or decoding of blocks using intra block copy ('BC') prediction. For example, some of the innovations relate to use of a default BV predictor with a non-zero value. Other innovations relate to use of a selected one of multiple BV predictor candidates for a current block. Still other innovations relate to use of a skip mode in which a current intra-BC-predicted block uses a predicted BV value.

Innovations in video playback using a browser-based video decoder are described. In a computer system that includes multiple central processing units ('CPUs'), a browser-based video decoder performs operations with multiple threads that may execute simultaneously on different CPUs. The video decoder can perform decoding operations in parallel for different sections of a picture. For example, with a main CPU thread associated with a browser, the video decoder performs a first decoding workload (e.g., bitstream parsing) for a picture. With auxiliary CPU threads associated with Web workers and simultaneously executing on different CPUs, the video decoder performs a second decoding workload (e.g., entropy decoding, decoding of side information) for different sections of the picture, one section per auxiliary CPU thread. If the computer system also includes a graphics processing unit ('GPU'), the video decoder can perform additional decoding workloads with shader routines executable on the GPU.

Browser-based video decoder using multiple CPU threads

Innovations in motion estimation adapted for screen remoting scenarios are described. For example, a video encoder calculates a hash value for a current block in a current picture. The video encoder searches, subject to a spatial constraint, for a matching block in a reference picture (e.g., the previous picture in display order) based at least in part on the hash value for the current block. The spatial constraint defines a search area in the reference picture within which hash values for candidate blocks in the reference picture may be compared to the hash value for the current block. By using a spatial constraint to limit the range of the local hash-based motion estimation, the video encoder can speed up the motion estimation process while still considering the candidate blocks in the reference picture that are most likely to match the current block.

Local hash-based motion estimation for screen remoting scenarios

Innovations in video decoding and rendering operations in a graphics pipeline, in which at least some of the operations are performed using a graphics processing unit ('GPU'), are described. For example, a video playback tool aggregates texture values for intra-coded blocks of a picture in central processing unit ('CPU') memory, then transfers the texture values for the intra-coded blocks from the CPU memory to GPU memory. The video playback tool performs operations to decode the encoded data and reconstruct the picture. For a given block (e.g., of a macroblock, coding unit) of the picture, a graphics primitive represents texture values for the given block as a point for processing by the GPU. The video playback tool uses one or more shader routines, executable by the GPU, to transfer texture values to a display buffer. In some cases, the video playback tool also performs decoding operations with the shader routines.

Efficient decoding and rendering of blocks in a graphics pipeline

Video frames of a higher-resolution chroma sampling format such as YUV 4:4:4 are packed into video frames of a lower-resolution chroma sampling format such as YUV 4:2:0 for purposes of video encoding. For example, sample values for a frame in YUV 4:4:4 format are packed into two frames in YUV 4:2:0 format. After decoding, the video frames of the lower-resolution chroma sampling format can be unpacked to reconstruct the video frames of the higher-resolution chroma sampling format. In this way, available encoders and decoders operating at the lower-resolution chroma sampling format can be used, while still retaining higher resolution chroma information. In example implementations, frames in YUV 4:4:4 format are packed into frames in YUV 4:2:0 format such that geometric correspondence is maintained between Y, U and V components for the frames in YUV 4:2:0 format.

Frame packing and unpacking higher-resolution chroma sampling formats

Innovations in motion estimation adapted for screen remoting scenarios are described herein. For example, as part of motion estimation for a current picture, a video encoder finds a pivot point in the current picture, calculates a hash value for the pivot point, and searches for a matching area in a previous picture. In doing so, the video encoder can calculate a hash index from the hash value and look up the hash index in a data structure to find candidate pivot points in the previous picture. The video encoder can compare the hash value for the pivot point in the current picture to a hash value for a candidate pivot point in the previous picture and, when the hash values match, compare sample values around the respective pivot points. In this way, the video encoder can quickly detect large areas of exact-match blocks having uniform motion.

Motion estimation for screen remoting scenarios

A GPU loads point sprites that represent coded blocks of transform coefficients of one or more frames encoded in a bitstream and loads a transform kernel as a transform kernel texture. The GPU constructs an output frame using an inverse transform on the coded blocks of transform coefficients by transforming the point sprites with the transform kernel texture and by optionally dequantizing the point sprites. A single render pass may be used in which the rasterization formula performs the inverse transform and optionally dequantization. To preserve bandwidth, a CPU may refrain from sending the GPU at least some zero valued transform coefficients for the point sprites. Also, to reduce processing, the transform coefficients can remain in a zig-zag arrangement. The transform kernel texture used in the decoding can correspond to a modified version of the basis matrices used to encode the frame, which compensates for the zig-zag arrangement.

Video decoding using point sprites

Innovations in video decoding and rendering operations for inter-coded blocks in a graphics pipeline, in which at least some of the operations are performed using a graphics processing unit ('GPU'), are described. For example, a video playback tool receives encoded data for a current picture and performs operations to decode the encoded data and reconstruct the current picture. For a given inter-coded block of the current picture, a graphics primitive represents texture values as a point for processing by the GPU. The graphics primitive can have one or more attributes, including a motion vector, a block size, a display index value (indicating a location in a display buffer), and/or a residual index value (indicating a location of residual values). The operations performed by the video playback tool can include interpolation of sample values at fractional-sample offsets and motion compensation performed for inter-coded blocks in multiple passes for different block sizes.

Efficient decoding and rendering of inter-coded blocks in a graphics pipeline

Variations of rho-domain rate control for video encoding or other media encoding are presented. For example, in some of the variations, an encoder sets a rho value for a unit of media based at least in part on a bit allocation for the unit. The encoder also computes transform coefficients for the unit using a frequency transform having multiple location-dependent scale factors, sets a value of quantization parameter ('QP') for the unit using a mapping of QP values to rho values, and uses the value of QP for the unit during quantization of the transform coefficients of the unit. When the QP-rho mapping is determined, a location-independent scale factor that approximates the multiple location-dependent scale factors is used and/or certain scaling operations are integrated, which reduces computational complexity while still supporting accurate rate control decisions. Implementations of such variations of rate control can exploit opportunities for caching and parallel computation.

Variations of rho-domain rate control

Innovations in encoding of video pictures in a high-resolution chroma sampling format (such as YUV 4:4:4) using a video encoder operating on coded pictures in a low-resolution chroma sampling format (such as YUV 4:2:0) are presented. For example, according to a set of decision rules, high chroma resolution details are selectively encoded on a region-by-region basis such that increases in bit rate (due to encoding of sample values for the high chroma resolution details) happen when and where corresponding increases in chroma resolution are likely to improve quality in noticeable ways. In this way, available encoders operating on coded pictures in the low-resolution chroma sampling format can be effectively used to provide high chroma resolution details.

Dynamically updating quality to higher chroma sampling rate

Described herein is providing GPU resources across machine boundaries for a virtual machine that migrates between servers. Data centers tend to have racks of servers that have limited access to GPUs. Accordingly, disclosed herein is providing GPU resources to computing devices that have limited access to GPUs across machine boundaries.

Delivering GPU resources to a migrating virtual machine

Frame packing and unpacking between frames of chroma sampling formats with different chroma resolutions

Growing community of inventors

Saratoga, CA, United States of America

B Anil Kumar