Microsoft Technology Licensing, LLC

Guosheng Sun

Lihua Zhu

B Anil Kumar

Olof L E Mases

Qi Zhang

Shir Aharon

Jie Yang

Gaoyuan Zhang

The present disclosure provides methods and apparatuses for inserting digital contents into a multi-view video. The multi-view video may comprise a plurality of views. At least one target region in the plurality of views may be identified. At least one digital content to be inserted may be determined. The multi-view video may be updated through adding the at least one digital content into the at least one target region.

Inserting digital contents into a multi-view video

Innovations in video decoding and rendering operations in a graphics pipeline, in which at least some of the operations are performed using a graphics processing unit ('GPU'), are described. For example, a video playback tool aggregates texture values for intra-coded blocks of a picture in central processing unit ('CPU') memory, then transfers the texture values for the intra-coded blocks from the CPU memory to GPU memory. The video playback tool performs operations to decode the encoded data and reconstruct the picture. For a given block (e.g., of a macroblock, coding unit) of the picture, a graphics primitive represents texture values for the given block as a point for processing by the GPU. The video playback tool uses one or more shader routines, executable by the GPU, to transfer texture values to a display buffer. In some cases, the video playback tool also performs decoding operations with the shader routines.

Efficient decoding and rendering of blocks in a graphics pipeline

A GPU loads point sprites that represent coded blocks of transform coefficients of one or more frames encoded in a bitstream and loads a transform kernel as a transform kernel texture. The GPU constructs an output frame using an inverse transform on the coded blocks of transform coefficients by transforming the point sprites with the transform kernel texture and by optionally dequantizing the point sprites. A single render pass may be used in which the rasterization formula performs the inverse transform and optionally dequantization. To preserve bandwidth, a CPU may refrain from sending the GPU at least some zero valued transform coefficients for the point sprites. Also, to reduce processing, the transform coefficients can remain in a zig-zag arrangement. The transform kernel texture used in the decoding can correspond to a modified version of the basis matrices used to encode the frame, which compensates for the zig-zag arrangement.

Video decoding using point sprites

Innovations in video decoding and rendering operations for inter-coded blocks in a graphics pipeline, in which at least some of the operations are performed using a graphics processing unit ('GPU'), are described. For example, a video playback tool receives encoded data for a current picture and performs operations to decode the encoded data and reconstruct the current picture. For a given inter-coded block of the current picture, a graphics primitive represents texture values as a point for processing by the GPU. The graphics primitive can have one or more attributes, including a motion vector, a block size, a display index value (indicating a location in a display buffer), and/or a residual index value (indicating a location of residual values). The operations performed by the video playback tool can include interpolation of sample values at fractional-sample offsets and motion compensation performed for inter-coded blocks in multiple passes for different block sizes.

Growing community of inventors

San Jose, CA, United States of America

Guosheng Sun