Apple Inc.

Alex Hayes

Graham D Finlayson

Ilya Romanenko

Han Gong

Existing digital image color style transfer encoding methods are often forced to compromise between performing artifact-free local image processing and performing sufficiently expressive global image processing. Moreover, such existing image style encoding methods may be under-constrained (thereby producing more visible artifacts in stylized output images than would be desired) or over-constrained (and thereby not being expressive enough to sufficiently learn and reproduce local/spatial color changes in the stylized output images). Thus, disclosed herein are techniques to combine the advantageous aspects of existing image stylization methods in a novel hybrid image processing method that effectively bifurcates the learning and reproduction of pixel luminance changes from the learning and reproduction of pixel color changes in stylized output images. Such techniques may also advantageously capture local color changes without introducing excessive compression artifacts in the generated stylized output images—while still performing the color style transfer image processing operations in a computationally-efficient fashion.

Photorealistic image color style transfer compression

A method of creating a multispectral decorrelation model for use in determining a visible image from a multispectral image captured using a multispectral image sensor, the method comprising the steps of: generating, using a plurality of quantum efficiency curves for the multispectral image sensor and a plurality of synthetic light spectrum vectors, a grid of synthetic multispectral pixel values and a corresponding grid of synthetic visible pixel values, wherein each synthetic visible pixel value is substantially decorrelated from a non-visible component of a corresponding synthetic multispectral pixel value; and determining a multispectral decorrelation model using the grid of synthetic multispectral pixel values and the corresponding grid of synthetic visible pixel values, wherein the multispectral decorrelation model in use maps a multispectral pixel value of the multispectral image to a visible pixel value of the visible image.

Multispectral image decorrelation method and system

Devices, methods, and non-transitory program storage devices are disclosed herein to provide improved multi-spectral image processing techniques for generating an enhanced output image, the techniques comprising: obtaining an N-channel (e.g., multispectral) input image; determining fusion weights and fallback weights (e.g., relative intensity weights) for each of the N-channels of the input image; blending the fusion and fallback weights based on an amount of gradient information to generate blended weights; modulating the blended weights for a plurality of frequency band representations of the input image; applying the modulated blended weights to the corresponding frequency band representations of the input image to generate a plurality of output image frequency band representations; producing an output luma image, based on the plurality of output image frequency band representations; and generating an output RGB image, based on the output luma image, which may then, e.g., be displayed to a user or stored to non-volatile memory.

Multispectral image processing system and method

Devices, methods, and non-transitory program storage devices for spatiotemporal image noise reduction are disclosed, comprising: maintaining an accumulated image in memory; and obtaining a first plurality of multispectral images (e.g., RGB-IR images). For each image in the first plurality of multispectral images, the method may: calculate a multispectral guide image for the current image; calculate blending weights for the current image; apply the calculated blending weights to each channel of the current image to generate a denoised current image; and update the accumulated image based on pixel differences between the denoised current image and the accumulated image. In some embodiments, additional images (e.g., the accumulated image and/or other images captured prior to or after a given current image) may also be included in the denoising operations for a given current image. Finally, the method may generate a denoised output image for each input image, based on the updated accumulated image.

Systems and methods of spatiotemporal image noise reduction for multispectral image data

A method and system for generating an output image from a plurality, N, of corresponding input image channels is described. A Jacobian matrix of the plurality of corresponding input image channels is determined. The principal characteristic vector of the outer product of the Jacobian matrix is calculated. The sign associated with the principal characteristic vector is set whereby an input image channel pixel projected by the principal characteristic vector results in a positive scalar value. The output image as a per-pixel projection of the input channels in the direction of the principal characteristic vector is generated.

Growing community of inventors

Ely, United Kingdom

Alex Hayes