Apple Inc.

Kaiming Liu

David R Pope

Maxim W Smirnov

Hui Chao

William T Warner

Jason R Holloway

Henryk K Blasinski

Embodiments relate to an image processing circuit that performs machine learning (ML) based noise reduction on image data. The image processing circuit includes a ML based noise reduction circuit that includes a hybrid kernel calculation circuit and a noise filtering circuit coupled to the hybrid kernel calculation circuit. The hybrid kernel calculation circuit generates, for each pixel of an image, a hybrid kernel by combining a ML kernel of each pixel of the image and a bilateral kernel of each pixel of the image. The noise filtering circuit performs, for each pixel of the image, noise filtering of the image using the hybrid kernel for each pixel of the image to generate a de-noised version of the image.

Machine learning based noise reduction circuit

Embodiments of the present disclosure relate performing registration of a first image to a second image where the first image is undistorted to linear space before applying a geometric transformation matrix to modify the first image to align with the second image. The geometric transformation matrix may be a linear matrix that causes the undistorted version of the first image to make translation movement, rotational movement or both. The undistorted and modified first image is then reverted back to nonlinear distorted space. Then the reverted first image may be warped to better align with the second image for fusing with the second image. In this way, visual distortions in the fused image such as wobbling may be reduced or eliminated.

Linear transform of undistorted image for fusion

Embodiments relate to an image processing circuit able to perform image fusion on received images in a first mode for fusing demosaiced and downscaled image data, and a second mode for fusing raw image data. In the first mode, raw image data received from an image sensor is demosaiced and resampled prior to undergoing image fusion. On the other hand, in the second raw image mode, the image processing circuit performs image fusion on the raw image data, and demosaics and resamples the generated fused raw image. The image processing circuit comprises an image fusion circuit that processes images of received image pyramids differently, based on whether the first or second mode is being run. The image fusion circuit further comprises an additional alpha-blending circuit allowing for the image fusion circuit to perform a fusion and an alpha-blending in a single pass through image fusion circuit.

Image fusion processor circuit for dual-mode image fusion architecture

A feature extractor determines reference feature locations from a portion of a reference image and corresponding feature locations from a portion of a warp image. A transform module determines a homography transform function that transforms versions of the corresponding feature locations to the reference feature locations. The homography transform function has an error below a threshold level, where the error represents a difference between the transformed corresponding feature locations and the reference feature locations. The local transform module generates transform parameters by processing the homography transform function. A warper circuit warps the portion of the warp image by at least applying the transform parameters to generate a portion of a warped image.

Growing community of inventors

Santa Clara, CA, United States of America

Kaiming Liu