Microsoft Technology Licensing, LLC

Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Cheng Chang

Chih-Lung Lin

Richard Lawrence Greene

Richard Webb

Thomas W Holcomb

Zhi Zhou

Shankar Lakshmi Regunathan

Stacey Spears

Ming-Chieh Lee

Shijun Sun

Dong Zheng

Weidong Zhao

Eric Christoffersen

Minghui Xia

Yuting Fan

Eugene Gorbatov

Chris Rojas

Yegnaswamy Sermadevi

Zhiqing Rao

Seokhoon Kim

A computing system may access first alpha values associated with first pixels in a first pixel region of an image and determine a bit budget for encoding the first alpha values. The computing system may then select a first alpha-encoding mode for the first alpha values to reflect a determination that the first alpha values are all fully transparent or all fully opaque, and encode the first alpha values by storing the selected first alpha-encoding mode as part of a metadata without using the bit budget to encode the first alpha values individually. The computing system may then update a record of unallocated bits available for allocation based on the bit budget unused in the encoding of the first alpha values, and allocate, based on the record of unallocated bits, bits to encode a set of alpha values different from the first alpha values.

Rate controlled image and texture data compression

A method by a rendering device includes receiving a request to render multiple surfaces corresponding to multiple virtual objects to be concurrently displayed on an augmented-reality (AR) headset. The method further includes that the AR headset is connected to the rendering device via a wireless link. In response to a determination that a network quality of the wireless link is below a threshold condition, the method further includes selecting a first subset of the multiple surfaces that are higher priority than a second subset of the plurality of surfaces. The method includes transmitting the first subset of multiple surfaces to the AR headset for display and transmitting the second subset of multiple surfaces to the AR headset for display after transmitting the first subset. This method includes rendering the surfaces in accordance with a set of rendering parameters so as to satisfy one or more network constraints.

Resilient rendering for augmented-reality devices

In one embodiment, a computing system may access color components of a pixel region in an image, and then determine a color variance for each of the color components. The computing system may further determine a desired bit allocation for each of the color components based on the color variance associated with that color component. The computing system may then determine a total bit allocation for the pixel region based on the desired bit allocations for the color components, as well as a number of unallocated bits available for allocation. The computing system may further determine a final bit allocation for each of the color components by allocating the total bit allocation to each of the color components according to the desired bit allocation for each of the color components. The computing system may then encode each of the color components using the associated final bit allocation.

In an embodiment, a method involves accessing a first pixel block of an image, the first pixel block comprising pixels, each associated with multiple pixel components, determining whether to separately or jointly encode the multiple pixel components of each of the pixels of the first pixel block, determining that the multiple pixel components of each of the pixels in the first pixel block are to be jointly encoded based on (1) determining, based on the multiple pixel components of each of the pixels, a line defined within a three-dimensional coordinate system in which each of the pixels is represented as a three-dimensional point and (2) determining that the line satisfies a predetermined criteria, and encoding the multiple pixel components of each of the pixels in the first pixel block as a single quantized value based on a projection of the three-dimensional point associated with that pixel onto the line.

Joint pixel and texture data compression

In one embodiment, a computing system may access first alpha values associated with first pixels in a first pixel region of an image and determine a bit budget for encoding the first alpha values. The computing system may then select a first alpha-encoding mode for the first alpha values to reflect a determination that the first alpha values are all fully transparent or all fully opaque, and encode the first alpha values by storing the selected first alpha-encoding mode as part of a metadata without using the bit budget to encode the first alpha values individually. The computing system may then update a record of unallocated bits available for allocation based on the bit budget unused in the encoding of the first alpha values, and allocate, based on the record of unallocated bits, bits to encode a set of alpha values different from the first alpha values.

Disclosed herein includes a system, a method, and a device for compressing image data. The device includes one or more processors, coupled to memory, configured to identify a plurality of sub-blocks of a block of image data including a first sub-block and a second sub-block. The one or more processors are configured to identify a first data characteristic of data of the first sub-block and a second data characteristic of data of the second sub-block, determine a first compression technique based at least on the first data characteristic of the first sub-block, determine a second compression technique based at least on the second data characteristic of the second sub-block, and compress the first sub-block using the first compression technique and the second sub-block using the second compression technique.

System and method for compressing and decompressing images using block-based compression format

In an embodiment, a method involves accessing a first pixel block of an image, the first pixel block comprising pixels, each associated with three color values, determining whether to separately or jointly encode the three color values of each of the pixels of the first pixel block, determining that the three color values of each of the pixels in the first pixel block are to be jointly encoded based on (1) determining, based on the three color values of each of the pixels, a line defined within a three-dimensional coordinate system in which each of the pixels is represented as a three-dimensional point and (2) determining that the line satisfies a predetermined criteria, and encoding the three color values of each of the pixels in the first pixel block as a single quantized value based on a projection of the three-dimensional point associated with that pixel onto the line.

Joint color image and texture data compression

In one embodiment, a computing system may access a first set of pixel values of an image having a bit depth. The first set of pixel values includes a first subset and a second subset. Each pixel value in the first subset is less than each pixel value in the second subset. The computing system may further determine an overflow range based on a pixel value overflow associated with the bit depth. The overflow range covers a second set of pixel values, which includes (1) the second subset and (2) an overflow representation of the first subset. In response to determining that the overflow range is less than a range covering the first set of pixel values, the computer system may then generate quantization levels corresponding to the overflow range and associate each pixel value of the second set of pixel values with one of the quantization levels.

Methods and systems for image data compression

Systems and methods for multi-layered rate control for scalable video coding. A parameter value may be calculated based on a current layer target bit rate and a current layer buffer state for a frame in a video stream. The frame may include a lower layer and one or more higher layers. A determination may then be made as to whether the current layer is the lower layer. If the current layer is the lower layer, a determination may then be made as to whether a coupling request has been received from a higher layer in the frame. If the coupling request has been received from the higher layer in the frame, the parameter value for the current layer may be increased based on a buffer state threshold value of the higher layer in the frame.

Multi-layered rate control for scalable video coding

Techniques and tools are presented for controlling artifacts such as banding artifacts, ringing artifacts and film scan artifacts in video. For example, before encoding, a pre-processor performs combined filtering and dithering on video such that the weight of dithering at a location depends on the results of filtering at the location. For the combined filtering and dithering, the pre-processor can determine a lowpass signal and highpass residual, weight dithering based on local characteristics of the highpass residual, and then combine the lowpass signal with the weighted dithering. Or, to determine the relative weight, the pre-processor can use a filter whose normalization factor varies depending on how many sample values around a location are within a threshold of similarity to a current sample value at the location. The filtering and dithering can use different strengths for luma and chroma channels.

Filtering and dithering as pre-processing before encoding

Techniques and tools are described for adaptive deadzone ('DZ') resizing during quantization. For example, in some embodiments, for quantization of an AC frequency coefficient of a block, a video encoder adjusts DZ size of a selected quantizer depending on the texture of the block. In other embodiments, a video encoder adjusts DZ size depending on the frequency of a coefficient being quantized. In still other embodiments, for quantization of an AC frequency coefficient of a block, a video encoder adjusts DZ size depending on the texture of the block and the frequency of the coefficient being quantized.

Adaptive deadzone size adjustment in quantization

Techniques and tools for selecting search ranges and/or motion vector ranges during motion estimation are described. For example, a video encoder performs motion estimation constrained by a first search range, which results in multiple motion vectors. The encoder computes motion vector distribution information for the motion vectors. To compute the distribution information, the encoder can track the motion vectors in a histogram and count how many of the motion vectors fall within each of multiple intervals for the distribution information. The encoder then selects a second search range and performs motion estimation constrained by the second search range. Selecting the second search range can include selecting a motion vector range, which in some cases in effect determines the second search range.

Dynamic selection of motion estimation search ranges and extended motion vector ranges

Techniques and tools for adaptive selection of picture quantization parameters ('QPs') for predicted pictures are described. For example, a video encoder adaptively selects a delta QP for a B-picture based on spatial complexity, temporal complexity, whether differential quantization is active, whether the B-picture is available as a reference picture, or some combination or subset of these or other factors. The delta QP can then be used to adjust the picture QP for the B-picture (e.g., to reduce bit rate for the B-picture without appreciably reducing the perceived quality of a video sequence.

Adaptive selection of picture-level quantization parameters for predicted video pictures

Techniques and tools are described for compensating for rounding when estimating sample-domain distortion in the transform domain. For example, a video encoder estimates pixel-domain distortion in the transform domain for a block of transform coefficients after compensating for rounding in the DC coefficient of the block. In this way, the video encoder improves the accuracy of pixel-domain distortion estimation but retains the computational advantages of performing the estimation in the transform domain. Rounding compensation includes, for example, looking up an index (from a de-quantized transform coefficient) in a rounding offset table to determine a rounding offset, then adjusting the coefficient by the offset. Other techniques and tools described herein are directed to creating rounding offset tables and encoders that make encoding decisions after considering rounding effects that occur after an inverse frequency transform on de-quantized transform coefficient values.

Estimating sample-domain distortion in the transform domain with rounding compensation

Multiple-pass video encoding systems and techniques are described which utilize statistics taken during a first-pass encoding to create complexity measurements for video data which is to be encoded. By analyzing these complexity measurements, preprocessing decisions, such as, for example, the determination of strength of denoise filters, can be made with greater accuracy. In one implementation, these complexity measurements take the form of calculation of temporal and spatial complexity parameters, which are then used to compute a unified complexity parameter for each group of pictures being encoded.

Growing community of inventors

Redmond, WA, United States of America

Cheng Chang