Qualcomm Incorporated

Narendranath Malayath

Haohong Wang

Kai Wang

Raghavendra Nagaraj

Sumit Mohan

Harinath Garudadri

Sharath Manjunath

Suhail Jalil

Chienchung Chang

Yan Ye

Ning Bi

Yingyong Qi

Pawan Kumar Baheti

Andrew P Dejaco

Naveen Srinivasamurthy

Yiliang L Bao

Stephen A Molloy

Xue Hai Fang

Mahant Siddaramanna

David Puig Oses

Anthony Patrick Mauro, Ii

Feng Ge

Aravind Bhaskara

King-Chung Lai

William Yee-Ming Huang

Hariharan Ganesh Lalgudi

Apoorva Nagarajan

Yong Chang

Jatinder Singh

Prasant Shekhar Singh

Systems, methods, and computer-readable media are provided for efficient video coding. An method can include determining, during a first coding stage implemented at a first frame rate, first motion vectors for a first subset of frames in a sequence of frames; determining, during a second coding stage, second motion vectors for a second subset of frames in the sequence of frames, wherein a portion of the second motion vectors is calculated based on one or more of the first motion vectors; and reconstruct, during the second coding stage, the first subset of frames using the first motion vectors; and reconstruct, during the second video coding stage, the second subset of frames using the second motion vectors, the first coding stage and the second coding stage being implemented in parallel, and the second coding stage being implemented at a second frame rate that is higher than the first frame rate.

Efficient video encoder architecture

Systems, methods, and computer-readable media are provided for efficient control and data utilization between processing components of a system. An method can include obtaining image data captured by an image sensor; prior to a first computing component performing a first set of operations on the image data and a second computing component performing a second set of operations on the image data, determining one or more common operations included in the first set of operations and the second set of operations, wherein the first set of operations is different than the second set of operations; performing the one or more common operations on the image data; and generating an output of the one or more operations for use by the first computing component to perform the first set of operations and the second computing component to perform the second set of operations.

Determining common operations for image processing computing components

Systems and techniques are described for data encoding using a machine learning approach to generate a distortion prediction {circumflex over (D)} and a predicted bit rate {circumflex over (R)}, and to use {circumflex over (D)} and {circumflex over (R)} to perform rate-distortion optimization (RDO). For example, a video encoder can generate the distortion prediction {circumflex over (D)} and the bit rate residual predictionbased on outputs of the one or more neural networks in response to the one or more neural networks receiving a residual portion of a block of a video frame as input. The video encoder can determine bit rate metadata predictionbased on metadata associated with a mode of compression, and determine {circumflex over (R)} to be the sum ofand. The video encoder can determine a rate-distortion cost prediction &#x134; as a function of {circumflex over (D)} and {circumflex over (R)}, and can determine a prediction mode for compressing the block based on &#x134;.

Machine learning based rate-distortion optimizer for video compression

Implementations include video image processing systems, methods, and apparatus for integrated video downscale in a video core. The downscaler computes and writes a display frame to an external memory. This frame may have the same resolution as a target display device (e.g., mobile device). The target display device then reads this display frame, rather than the original higher resolution frame. By enabling downscale during encoding/decoding, the device can conserve resources such as memory bandwidth, memory access, bus bandwidth, and power consumption associated with separately downscaling a frame of video data.

Integrated downscale in video core

The motion estimation techniques and video encoding device(s) described use a two dimensional pipeline to generate accurate motion estimation parameters for a current video block. The two dimensional pipeline uses previously calculated motion estimation parameters of relevant neighboring video blocks, including a preceding video block on a same row as the current video block, prior to the generation of the accurate motion vectors, motion vector predictors, and mode decision of the current video block. The use of the two dimensional pipeline allows accurate motion vector prediction from neighboring video blocks previously not available, in the computation of motion vectors. Three engines may be used in the two dimensional pipeline, a fetch engine, an integer search engine and a fractional and spatial search engine. While the fetch engine and fraction and spatial search engine operate on one row, the integer search engine operates on another row.

Multi-dimensional neighboring block prediction for video encoding

Techniques for estimating distortion due to quantization of data are described. A histogram with multiple bins may be obtained for a set of coefficients to be quantized. Distortion due to quantization of the set of coefficients may be estimated based on the histogram and average distortions for the histogram bins. The number of coefficients in each bin may be multiplied with an average distortion for the bin to obtain a per-bin distortion. The per-bin distortions for all of the bins may be accumulated and scaled with a correction factor to obtain the estimated distortion. The techniques may be used to estimate distortions for a set of coding elements. Distortion and rate may be estimated for each coding element for each of multiple quantization steps. A set of quantization steps may be selected for the set of coding elements based on the estimated distortions and the estimated rates for the set of coding elements for different quantization steps.

Distortion estimation for quantized data

Techniques for intensity compensation in video processing are provided. In one configuration, a wireless communication device compliant with the VC1-SMPTE standard (e.g., cellular phone, etc.) comprises a processor that is configured to execute instructions operative to reconstruct reference frames from a received video bitstream. A non-intensity-compensated copy of a reference frame of the bitstream is stored in a memory of the device and used for defining the displayable images and for on-the-fly generation of a stream of intensity-compensated pixels to perform motion compensation calculations for frames of the video bitstream.

Intensity compensation techniques in video processing

Motion estimation in video compressions systems. A programmable motion estimator may be used to estimate a motion vector for a macroblock in a current frame by searching for a matching macroblock in a previous frame. A controller may be used to program the motion estimator to perform a particular search.

Motion estimation in video compression systems

This disclosure describes rate control techniques that can improve video coding based on a 'two-pass' approach. The first pass codes a video sequence using a first set of quantization parameters (QPs) for the purpose of estimating rate-distortion characteristics of the video sequence based on the statistic of the first pass. A second set of QPs can then be defined for a second coding pass. The estimated rate-distortion characteristics of the first pass are used to select Qps for the second pass in a manner that minimizes quality fluctuation between the frames of the video sequence. Furthermore, selection of the second set of QPs may also substantially maximize quality of the frames at the substantially minimized quality flucuation in order to achieve low average frame distortion with the minimized quality fluctuation.

Two pass rate control techniques for video coding using a min-max approach

The disclosure describes FGS video coding techniques that use cycle-aligned fragments (CAFs). The techniques may perform cycle-based coding of FGS video data block coefficients and syntax elements, and encapsulate cycles in fragments for transmission. The fragments may be cycle-aligned such that a start of a payload of each of the fragments substantially coincides with a start of one of the cycles. In this manner, cycles can be readily accessed via individual fragments. Some cycles may be controlled with a vector mode to scan to a predefined position within a block before moving to another block. In this manner, the number of cycles can be reduced, reducing the number of fragments and associated overhead. The CAFs may be entropy coded independently of one another so that each fragment may be readily accessed and decoded without waiting for decoding of other fragments. Independent entropy coding may permit parallel decoding and simultaneous processing of fragments.

Video coding with fine granularity scalability using cycle-aligned fragments

An embodiment is directed to a method for selecting a predictive macroblock partition from a plurality of candidate macroblock partitions in motion estimation and compensation in a video encoder including determining a bit rate signal for each of the candidate macroblock partitions, generating a distortion signal for each of the candidate macroblock partitions, calculating a cost for each of the candidate macroblock partitions based on respective bit rate and distortion signals to produce a plurality of costs, and determining a motion vector from the costs. The motion vector designates the predictive macroblock partition.

Video encoding

This disclosure describes identifying key frames from a sequence of video frames. A first set of information generated by operating on uncompressed data is accessed. A second set of information generated by compressing the data is also accessed. The first and second sets of information are used to identify key frames from the video frames.

Selecting key frames from video frames

This disclosure describes rate control techniques that can improve video coding based on a 'two-pass' approach. The first pass codes a video sequence using a first set of quantization parameters (QPs) for the purpose of estimating rate-distortion characteristics of the video sequence based on the statistics of the first pass. A second set of QPs can then be defined for a second coding pass. The estimated rate-distortion characteristics of the first pass are used to select QPs for the second pass in a manner that minimizes distortion of the frames of the video sequence.

Two pass rate control techniques for video coding using rate-distortion characteristics

The disclosure is directed to scalable motion estimation techniques for video encoding. According to the motion estimation techniques, a motion vector search is scaled according to the computing resources available. For example, the extent of the search may be dynamically adjusted according to available computing resources. A more extensive search may be performed when computing resources permit. When computing resources are scarce, the search may be more limited. In this manner, the scalable motion estimation technique balances video quality, computing overhead and power consumption. The scalable motion estimation technique may search a series of concentric regions, starting at a central anchor point and moving outward across several concentric regions. The number of concentric regions searched for a particular video frame or macroblock is adjusted according to computing resources. Upon searching the anchor point, the search proceeds outward to the next concentric region, and continues as permitted by available computing resources.

Scalable motion estimation for video encoding

The disclosure is directed to video processing. The various video processing techniques include generating blocks of information for a frame of video, allocating bits from a bit budget to each of the blocks, the number of bits being allocated to each of the blocks being a function of the information contained therein, and using the bits allocated to each of the blocks to represent the information contained therein.

Growing community of inventors

San Diego, CA, United States of America

Narendranath Malayath