Nvidia Corporation

Omnivision Technologies, Inc.

Intel Corporation

Yongshen Ni

Eric Dujardin

Jizhang Shan

Guansong Liu

Hsin-I Cindy Liu

Chengwu Cui

Houxin Zhang

In various examples, systems and methods for calibration for sensor lens shading using non-radial correction of residual radial shading error are provided. In some embodiments, a calibration flow includes computation of calibration parameters corresponding to radial lens shading correction, and computation of calibration parameters corresponding to non-radial lens shading correction. A lens shading profile may be computed that defines a gain mapping of lens shading effect appearing in an image frame of calibration sensor data. Parameters for radial lens shading correction may be computed from the lens shading profile, and parameters for non-radial lens shading correction may be computed based a residual shading profile generated from the radial lens shading correction. Calibration parameters for radial and non-radial lens shading correction may be used to calibrate sensor data captured by an image sensor module to correct for lens shading.

Shading calibration for radial sensor lenses

In various examples, image processing techniques are presented for reducing artifacts in images for autonomous or semi-autonomous systems and applications. Systems and methods are disclosed for deferring at least a portion of a strength associated with a color correction matrix (CCM) of an image processing pipeline to one or more other stages associated with the image processing pipeline. For instance, the CCM is used to determine a first CCM and a second, deferred CCM. The first CCM may be associated with an earlier stage of the image processing pipeline while the deferred CCM is associated with a later stage of the image processing pipeline. In some examples, the deferred CCM is combined with another matrix, such as a color space conversion matrix. By deferring part of the CCM, the image processing pipeline may reduce a number of artifacts and/or eliminate the artifacts that occur when processing image data.

Deferred color correction in image processing pipelines for autonomous systems and applications

In various examples, lens shading image correction systems and applications using non-radial correction of residual radial shading error are provided. In some embodiments, lens shading image correction may be implemented using calibration parameters corresponding to radial lens shading correction, and calibration parameters corresponding to non-radial lens shading correction. In some embodiments, sensor data comprising an image frame may be captured using a sensor. Radial lens shading correction may be applied to the image frame to produce a residual shading profile, and non-radial lens shading correction may be applied to the residual shading profile to produce a calibrated image frame. Parameters for radial lens shading correction may be computed from a lens shading profile associated with the sensor, and parameters for non-radial lens shading correction may be computed based a residual shading profile produced from the radial lens shading correction.

Lens shading using non-radial image correction

Techniques are provided for color correction of image frames, received from a hybrid RGBIR sensor, using resolution recovery of the red, green, blue (RGB), and infra-red (IR) channels. The color correction is to compensate for IR contamination of the RGB channels. A methodology implementing the techniques according to an embodiment includes a demosaic operation on the RGBIR image frame to generate full resolution red, green, blue, and IR frames. The demosaic operation includes filtering to extract Chroma and Luma components of the RGBIR image to reconstruct the full resolution images. The method also includes calculating an IR weighting factor for the pixels of each of the full resolution RGB frames, and correcting for IR contamination of those pixels by subtracting a scaled value of the corresponding pixels from the full resolution IR frame. The scaled value is based on the IR weighting factor associated with the pixel to be corrected.

Color correction of RGBIR sensor stream based on resolution recovery of RGB and IR channels

A method for processing a plurality of images of a scene recorded from different vantage points, where the plurality of images includes a color reference image captured by a Bayer type camera and at least one additional image, the method including (a) registering at least a portion of the plurality of images, and (b) generating a unitary color image from the plurality of images, wherein color information of the unitary color image is determined exclusively from the color reference image.

Camera array systems including at least one bayer type camera and associated methods

An image sensor apparatus is disclosed. The image sensor apparatus includes an image sensor for generating image data corresponding to an optical image. The image sensor apparatus also includes a color filter customized for a lens distortion model. A processor processes the image data with a plurality of distortion correction routines to generate a digital image.

Growing community of inventors

San Jose, CA, United States of America

Yongshen Ni