Apple Inc.

Aptina Imaging Corporation

Semiconductor Components Industries, LLC

Chihsin Wu

David R Pope

Sheng Lin

Maxim W Smirnov

Frederic Cao

Touraj Tajbakhsh

Amnon D Silverstein

Dan C Lelescu

D Amnon Silverstein

Oren Kerem

Elena Lamburn

Cheng Lu

A foveated down sampling and correction (FDS-C) circuit for combined down sampling and correction of chromatic aberrations in images. The FDS-C circuit performs down sampling and interpolation of pixel values of a first subset of pixels of a color in a raw image using down sampling scale factors and first interpolation coefficients to generate first corrected pixel values for pixels of the color in a first corrected version of the raw image. The FDS-C circuit further performs interpolation of pixel values of a second subset of the pixels in the first corrected version using second interpolation coefficients to generate second corrected pixel values for pixels of the color in a second corrected version of the raw image. Pixels in the first subset are arranged in a first direction, pixels in the second subset are arranged in a second direction, and the down sampling scale factors vary along the first direction.

Circuit for combined down sampling and correction of image data

An image processing circuit for performing optical image stabilization before lens shading correction. The image processing circuit determines, for each row of pixels in an image captured through an image sensor receiving light through a lens having an optical center with a first offset along a first direction and a second offset along a second direction orthogonal to the first direction, a first grid offset along the first direction using the first offset and a coordinate of that row of pixels along the second direction. The image processing circuit determines, for each row of pixels in the image, a second grid offset along the second direction using the second offset and the coordinate of that row of pixels along the second direction. The image processing circuit generates an adjusted grid for the pixels of the image using the first grid offset and the second grid offset.

Circuit for performing optical image stabilization before lens shading correction

Embodiments relate to lateral chromatic aberration (LCA) recovery of raw image data generated by image sensors. A chromatic aberration recovery circuit performs chromatic aberration recovery on the raw image data to correct the resulting LCA in the full color images using pre-calculated offset values of a subset of colors of pixels.

Circuit for correcting lateral chromatic abberation

Embodiments relate to hue preservation post processing for highlight recovery of an input image. Intensity values for multiple color channels of a plurality of color channels of a pixel of the input image is determined using corresponding ratios of target hues for the plurality of color channels of the pixel, wherein the pixel has at least one color channel with an intensity above a predetermined threshold. A hue preserved value for a color channel of the plurality of color channels of the pixel is determined using intensity values determined for the plurality of color channels of the pixel and the target hues. A recovered version of the input image is generated by adjusting hue information of the pixel, using the hue preserved value for the channel of the plurality of color channels of the pixel.

Hue preservation post processing with early exit for highlight recovery

Embodiments relate to processing of pixels captured by a quadra image sensor. A quadra image sensor includes a plurality of pixel tiles, each having a plurality of pixels corresponding to the same color. A lens shading correction (LSC) circuit receives, for each of a plurality of colors, a set of gain tables. Each gain table corresponds to a different channel associated with the color. Each gain table includes a set of gain values, each associated with a location. The LSC circuit determines a pixel gain value for each pixel in a pixel tile and scales the pixels in the pixel tile based on the determined pixel gain values.

Lens shading correction for quadra image sensors

Embodiments relate to pixel conversion of images for display. A circuit converts input pixel values of an image using a color conversion function. The circuit is operable in different modes where each mode uses a different color conversion function. A lookup table memory circuit stores a mapping of color converted values and input pixel values according to the mode of operation where the mapping represents the color conversion function associated with the mode. The circuit produces a color converted value from the lookup table as a color converted version of a first input pixel value responsive to the first input pixel value being within a first range. The circuit may also produce a color converted version of a second input pixel value by interpolating a subset of the color converted values received from the lookup table responsive to the second input pixel being within a second input range.

High dynamic range color conversion using selective interpolation for different curves

Hue preservation post processing for highlight recovery

A portable electronic device may include an image signal processor that includes a clipping circuit, a pyramid generator circuit, and an image fusion processor. The clipping circuit clips pixel values that are under-exposed or over-exposed. The pyramid generator circuit applies a filter to the pixels of the image to generate a filtered image. Some of the filtered pixels may be generated from one or more clipped pixel values. The pyramid generator circuit identifies those filtered pixels that are generated from one or more clipped pixel values and marks the identified filtered pixels with a tag. The pyramid generator circuit decimates the filtered image to generate a downscaled image, which may include one or more filtered pixels that are marked with the tags. The image fusion processor fuses the downscaled image with another image. The pixels that are marked with the tags may be disregarded in the fusion process.

Tagging clipped pixels for pyramid processing in image signal processor

Embodiments relate to circuitry for pixel conversion of images for display. A circuit converts input pixel values of an image using a color conversion function. A lookup table memory circuit stores a mapping of color converted values and input pixel values where the mapping represents the color conversion function. The circuit produces a color converted value from the lookup table as a color converted version of a first input pixel value responsive to the first input pixel value being within a first range. The circuit may also produce a color converted version of a second input pixel value by interpolating a subset of the color converted values received from the lookup table responsive to the second input pixel being within a second input range.

High dynamic range color conversion using selective interpolation

Embodiments relate to circuitry for performing fusion of two images captured with two different exposure times to generate a fused image having a higher dynamic range. Information about first keypoints is extracted from the first image by processing pixel values of pixels in the first image. A model describing correspondence between the first image and the second image is then built by processing at least the information about first keypoints. A processed version of the first image is warped using mapping information in the model to generate a warped version of the first image spatially more aligned to the second image than to the first image. The warped version of the first image is fused with a processed version of the second image to generate the fused image.

Image fusion architecture

Systems and methods are provided for an adjustable filter engine. In particular, an electronic system is provided that can include a focus module, memory, and control circuitry. In some embodiments, the focus module can include an adjustable filter engine and a motor. By using the adjustable filter engine to generate a filter with a large number of filter coefficients, the control circuitry can accommodate a variety of system characteristics. For example, by generating a set of cumulative coefficients and re-arranging the order of the cumulative coefficients, the control circuitry can reduce the bit-width requirements of the adjustable filter engine hardware. For instance, the control circuitry can reduce the number of multipliers required to perform a convolution between an updated filter and one or more input signals. In some embodiments, the updated filter can be generated to reduce oscillations of the motor movement due to a new position request.

Systems and methods for an adjustable filter engine

Systems and methods are provided for focus recovery of multi-channel images. Control circuitry of an imaging system can restore an image by removing image blurring introduced by the lens, sensor noise introduced by the sensor, and a signal offset between multiple channels of the image. In some embodiments, the control circuitry can calculate one or more estimates of a signal offset of multiple observed signals. Using statistics generated from offset-removed signals, the control circuitry can generate one or more recovery kernels which can be applied to offset-removed signals to generate recovered signals. In other embodiments, instead of explicitly removing a signal offset from each observed signal, the control circuitry can implicitly remove the signal offset when calculating the first and second order statistics of one or more observed signals.

Growing community of inventors

San Jose, CA, United States of America

Chihsin Wu