Meta Platforms, Inc.

Ambarella International LP

Ambarella, Inc.

Junqiang Lan

Harikrishna Madadi Reddy

Sujith Srinivasan

Yunqing Chen

Guogang Hua

Chung-Fu Lin

Gaurang Chaudhari

Zhao Wang

Baheerathan Anandharengan

Srikanth Alaparthi

Xianliang Zha

Zhijian Lu

Prahlad Rao Venkatapuram

Zhijun Lei

Xing Cindy Chen

Kameswara Kishore Sriadibhatla

Adrian Stafford Lewis

Anil Muthiraparampil Sunil

A video to be encoded using a codec is received Pre-filter encoding of a specific frame of the video is performed. In a single-pass processing of the specific frame, frame-level filter parameters determined using a previous frame are used to apply an in-loop filter on blocks of the specific frame during encoding of the specific frame. Statistics for the specific frame are gathered for determining frame-level filter parameters to be used for a future frame of the video. Post-filter encoding of the specific frame of the video is performed.

Optimization of constrained directional enhancement filtering (CDEF)

A video encoder is disclosed. The video encoder comprises an integer level motion estimation hardware component configured to determine candidate integer level motion vectors for a video being encoded. The video encoder comprises a fractional motion estimation hardware component configured to receive the candidate integer level motion vectors from the integer level motion estimation hardware component and refine the candidate integer level motion vectors into candidate sub-pixel level motion vectors. The fractional motion estimation hardware component includes parallel pipelines configured to process coding units of a frame of the video in parallel across the parallel pipelines.

Fractional motion estimation engine with parallel code unit pipelines

A disclosed system may include a hardware distortion data pipeline that may include (1) a quantization module that generates a quantized data set, (2) an inverse quantization module that generates, from the quantized data set, an inverse quantized data set by executing an inverse quantization of the quantized data set, and (3) an inverse transformation module that generates an inversely transformed data set by executing an inverse transformation of the inverse quantized data set. The system may also include a hardware determination pipeline that determines a distortion metric based on the inversely transformed data set and the residual frame data set, and a hardware token rate pipeline that determines, based on the quantized data set, a token rate for an encoding of the residual frame data set via a video encoding pipeline. Various other methods, systems, and computer-readable media are also disclosed.

Hardware pipelines for rate-distortion optimization (RDO) that support multiple codecs

A disclosed computer-implemented method may include (1) selecting, from a video stream, a reference frame and a current frame, (2) collecting a reference histogram of the reference frame and a current histogram of the current frame, and (3) generating a smoothed reference histogram by applying a smoothing function to at least a portion of the reference histogram. In some examples, the computer-implemented method may also include (1) determining a similarity metric between the smoothed reference histogram and the current histogram and, (2) when the similarity metric is greater than a threshold value, applying weighted prediction during a motion estimation portion of an encoding of the video stream. Various other methods, systems, and computer-readable media are also disclosed.

Systems and methods for histogram-based weighted prediction in video encoding

A system comprises a memory storage configured to store at least a portion of a frame of a video and a hardware motion estimation search processing unit configured to perform at least a portion of a motion estimation search for the video for a plurality of different block sizes. The hardware motion estimation search processing unit is configured to perform the motion estimation search using a plurality of source sub-blocks of a first block size to determine a first type of comparison evaluation values for the first block size. A combination of values included in the first type of comparison evaluation values is utilized to determine at least one second type of comparison evaluation values for a second block size, wherein the second block size is larger than the first block size.

Unified search window to support multiple video encoding standards

A system for calculating token rates for video encoding includes a plurality of different probability lookup tables implemented in hardware, wherein each of the probability lookup tables specifically corresponds to a different prediction mode of a video codec. The system includes an application-specific integrated circuit compute unit. For each candidate prediction mode among the different prediction modes, the application-specific integrated circuit is configured to determine a rate distortion cost (RD Cost) for a video. The application-specific integrated circuit is configured to select one of the plurality of different probability lookup tables that corresponds to the candidate prediction mode and use the selected one of the plurality of different probability lookup tables to calculate a corresponding token rate for the candidate prediction mode. The application-specific integrated circuit is configured to encode the video using a selected one of the different prediction modes determined based on the determined rate distortion costs.

Hardware optimization of rate calculation in rate distortion optimization for video encoding

A system comprises a source block buffer and a plurality of hardware motion estimation search processing units in communication with the source block buffer. The source block buffer is configured to store at least a portion of a source block of a source frame of a video. The plurality of hardware motion estimation search processing units are configured to perform at least a portion of a motion estimation for the source block at least in part in parallel across a plurality of different reference frames of the video. Each of the hardware motion estimation search processing units is configured to be assigned a different one of the plurality of different reference frames and is configured to compare at least the portion of the source block with a portion of the assigned one of the different reference frames.

Hierarchical motion search processing

An apparatus includes a first memory interface circuit and a decompression circuit coupled to the first memory interface circuit. The decompression circuit may be configured to (i) receive a reduced size representation of a coding block of data comprising a first bit map, a second bit map, and zero or more non-zero values from an external memory via the first memory interface circuit, (ii) losslessly restore the coding block of data from the reduced size representation of the coding block using the first bit map, the second bit map, and the zero or more non-zero values, and (iii) transfer the restored coding block of data to a processing circuit.

High throughput hardware unit providing efficient lossless data compression in convolution neural networks

An apparatus includes a first memory interface circuit, a second memory interface circuit, and a compression circuit coupled between the first memory interface circuit and the second memory interface circuit. The compression circuit may be configured to receive a coding block of data via the first memory interface circuit, generate a reduced size representation of the coding block, and write the reduced size representation of the coding block to an external memory using the second memory interface circuit. The reduced size representation of said coding block generally comprises a first bit map, a second bit map, and zero or more non-zero values.

Growing community of inventors

Fremont, CA, United States of America

Junqiang Lan