Qualcomm Incorporated

Sigma Designs, Inc.

Alireza Shoa Hassani Lashdan

Darren Gnanapragasam

Stone Yun

David Hansen

Evgenii Krasnikov

Adrian Leung

Ike Ikizyan

Haoping Xu

Jeffrey Kar Fai Wong

Sivakumar Chidambaram

Louie Lee Toy

Gregory Vansickle

Simiao Wu

Xiaojian Liu

Arshia Ershadi

Hooshmand Shokri Razaghi

Justin Wong

Qureshi Asma

The present disclosure relates to methods and devices for display processing including an apparatus, e.g., a DPU. The apparatus may detect at least one of a scene change between successive frames of a plurality of frames or a threshold number of received frames of the plurality of frames. The apparatus may also generate at least one of a saliency map, an object segmentation map, or a depth map based on a down-sampled image of a first frame. The apparatus may also apply a CMF for a color space associated with the plurality of frames to the plurality of pixels in one or more subsequent frames of the plurality of frames, the CMF being applied until at least one of a subsequent scene change or a subsequent threshold number of received frames.

Methods and apparatus for saliency based frame color enhancement

Techniques and systems are provided for image processing. For instance, a process can include generating motion vectors representing differences between a past image of a video and a current image of a video, wherein the current image occurs after the past image in the video, wherein the motion vectors are generated using a first machine learning model; applying the motion vectors to a third image to generate a warped third image; generating blending weights based on a comparison between the warped third image and the current image, wherein the blending weights are generated using a second machine learning model; upscaling the current image to generate an upscaled current image, wherein the upscaled current image is upscaled by a third machine learning model; combining the upscaled current image and a warped fourth image using the blending weights to generate an upscaled output image; and outputting the upscaled output image.

Machine learning based temporal stabilization of images

Systems and techniques are described herein for processing video data. For instance, a technique can include receiving a first image after a previous image. The process can further include receiving a first segmentation mask associated with the previous image. The process can also include estimating a first set of forward motion vectors between the previous image and the first image. The process can further include estimating a reliability of the first set of forward motion vectors. The process can also include extrapolating a second segmentation mask associated with the first image using the first set of forward motion vectors and first segmentation mask based on the estimated reliability of the first set of forward motion vectors.

Segmentation mask extrapolation

Technologies are provided for processing data in neural networks. An example method can include processing, by each layer of a neural network, a row in a first stripe of a data input, the row being processed sequentially in a horizontal direction and according to a layer-by-layer sequence; after processing the row, processing, by each layer, subsequent rows in the first stripe on a row-by-row basis, each subsequent row being processed sequentially in the horizontal direction and according to the layer-by-layer sequence; generating an output stripe based on the processing of the row and subsequent rows; processing, by each layer, a second stripe of the data input, each row in the second stripe being processed in the horizontal direction and according to the layer-by-layer sequence, wherein rows in the second stripe are processed on a row-by-row basis; and generating another output stripe based on the processing of the second stripe.

Processing data in pixel-to-pixel neural networks

A device includes a memory and one or more processors. The memory is configured to store an image enhancement network of an image enhancer. The one or more processors are configured to predict an image compression quality of an image of a stream of images. The one or more processors are also configured to configure the image enhancer based on the image compression quality. The one or more processors are further configured to process, using the image enhancement network of the configured image enhancer, the image to generate an enhanced image.

Configurable image enhancement

A device includes a memory and one or more processors. The memory is configured to store instructions. The one or more processors are configured to execute the instructions to apply a neural network to a first image to generate an enhanced image. The one or more processors are also configured to execute the instructions to adjust at least a portion of a high-frequency component of the enhanced image based on a motion compensation operation to generate an adjusted high-frequency image component. The one or more processors are further configured to execute the instructions to combine a low-frequency component of the enhanced image and the adjusted high-frequency image component to generate an adjusted enhanced image.

Motion compensation for neural network enhanced images

A device includes one or more processors configured to execute instructions to obtain motion data indicating estimated motion between a first frame and a second frame of an input sequence of image frames, and to identify, based on the motion data, any frame regions of the first frame that indicate motion greater than a motion threshold. The one or more processors are also configured to determine, based on the motion data, a motion metric associated with the identified frame regions, and to perform a determination, based on the motion metric and a size metric associated with the identified frame regions, whether to use motion-compensated frame interpolation to generate an intermediate frame. The one or more processors are further configured to generate the intermediate frame based on the determination, and to generate an output sequence of image frames that includes the intermediate frame between the first frame and the second frame.

Selective motion-compensated frame interpolation

A device includes a memory configured to store an adapted network that is configured to generate a modified image based on a single image. The device includes a processor configured to obtain, from a stream of video data, a first distorted image depicting an object, and to provide the first distorted image to the adapted network to generate a first modified image. The processor is configured to obtain, from the stream of video data, a second distorted image depicting the object, and to provide the second distorted image to the adapted network to generate a second modified image. The object is distorted differently in the second distorted image than in the first distorted image. The processor is configured to generate a video output including the first modified image and the second modified image without visible artifacts due to distortion differences between the first distorted image and the second distorted image.

System and method to process images of a video stream

A camera image processor receives frames from an image sensor of the camera. While continuing to receive frames from the image sensor, the camera image processor detecting an action within one or more frames of the received frames, for example using a convolution neural network trained to recognize one or more actions. While continuing to receive frames from the image sensor, the camera image processor captures the one or more frames with the detected action, for example as a still image, or as a slow-motion portion of a video that includes the received frames.

Image capture based on action recognition

Methods, devices, and systems for simulating motion blur are disclosed. In some aspects, a device includes a memory and one or more processors coupled to the memory. The one or more processors may be configured to receive a plurality of frames, identify an object of interest within the plurality of frames, track the object of interest within the plurality of frames, align the object of interest within the plurality of frames, and generate a final frame based on blending the aligned plurality of frames.

Motion blur simulation

A device obtains a plurality of original high-resolution frames and a low-resolution frame. The device generates, based on the plurality of original high-resolution frames, a first additional high-resolution frame. The first additional high-resolution frame and the low-resolution frame correspond to a same output time. The device generates a down-sampled frame by down-sampling the first additional higher-resolution frame. Additionally, the device determines, based on comparisons of blocks of the low-resolution frame and blocks of the down-sampled frame, a plurality of weights. The device generates an up-sampled frame by up-sampling the low-resolution frame. The device generates a second additional high-resolution frame based on a weighted average of the up-sampled frame and the first additional high-resolution frame. The weighted average is based on the plurality of weights. The second additional high-resolution frame corresponds to the same output time as the first additional high-resolution frame and the low-resolution frame.

Using low-resolution frames to increase frame rate of high-resolution frames

To reduce artifacts caused by quantization errors in image compression systems, an offset is added to quantized samples at low frequency sections of a macroblock. A decoder uses the offset after decoding to bring the decoded sample closer to the original pre-encoded and pre-quantized sample, thereby compensating for quantization errors.

System and method for scalar quantization error reduction

A system and method combine an entropy coding algorithm like Huffman or arithmetic coding with a fixed length coding scheme and can improve the compression performance in a fixed rate compression scheme. The fixed length code assigns codes with a fixed length to symbols that have the highest probability of occurrence. Therefore, fixed length coding is used if all symbols in the sequence are from the set of symbols that have the highest probability values. Otherwise an entropy coding algorithm (e.g. Huffman coding) is used to encode quantized symbols.

Growing community of inventors

Burlington, Canada

Alireza Shoa Hassani Lashdan