Adobe, Inc.

Google Inc.

Zhihao Xia

Michael Gharbi

Kalyan Krishna Sunkavalli

Federico Perazzi

Supreeth Achar

Jason Lawrence

Elya Shechtman

He Zhang

Peter Merrill

Yifei Fan

Ke Wang

Jiawen Chen

Ethan Tseng

Lars Jebe

Digital content stylization techniques are described that leverage a neural photofinisher to generate stylized digital images. In one example, the neural photofinisher is implemented as part of a stylization system to train a neural network to perform digital image style transfer operations using reference digital content as training data. The training includes calculating a style loss term that identifies a particular visual style of the reference digital content. Once trained, the stylization system receives a digital image and generates a feature map of a scene depicted by the digital image. Based on the feature map as well as the style loss, the stylization system determines visual parameter values to apply to the digital image to incorporate a visual appearance of the particular visual style. The stylization system generates the stylized digital image by applying the visual parameter values to the digital image automatically and without user intervention.

Neural photofinisher digital content stylization

An image processing system employs a parametric model for image harmonization of composite images. The parametric model employs a two-stage approach to harmonize an input composite image. At a first stage, a color curve prediction model predicts color curve parameters for the composite image. At a second stage, the composite image with the color curve parameters is input to a shadow map prediction model, which predicts a shadow map. The predicted color curve parameters and shadow map are applied to the composite image to provide a harmonized composite image. In some aspects, the color curve parameters and shadow map are predicted using a lower-resolution composite image and up-sampled to apply to a higher-resolution version of the composite image. The harmonized composite image can be output with the predicted color curve parameters and/or shadow map, which can be modified by a user to further enhance the harmonized composite image.

Parametric composite image harmonization

An imaging system includes a processor, a memory, a visible light camera configured to record a first image of a scene, and an infrared camera configured to record a second image of the scene. The processor configured to execute instructions stored in the memory to input the first image and the second image into a neural network. The neural network relights the first image, based on characteristics of the second image, to correspond to an image of the scene under canonical illumination conditions.

Infrared light-guided portrait relighting

Techniques of estimating surface normals and reflectance from poorly-lit images includes using, in addition to an RGB image of a subject of a set of subjects, an image illuminated with near-infrared (NIR) radiation to determine albedo and surface normal maps for performing an image relighting, the image being captured with the NIR radiation from essentially the same perspective from which the RGB image was captured. In some implementations, a prediction engine takes as input a single RGB image and a single NIR image and estimates surface normals and reflectance from the subject.

Dark flash normal camera

Certain embodiments involve techniques for efficiently estimating denoising kernels for generating denoised images. For instance, a neural network receives a noisy reference image to denoise. The neural network uses a kernel dictionary of base kernels and generates a coefficient vector for each pixel in the reference image such that the coefficient vector includes a coefficient value for each base kernel in the kernel dictionary, where the base kernels are combined to generate a denoising kernel and each coefficient value indicates a contribution of a given base kernel to a denoising kernel. The neural network calculates the denoising kernel for a given pixel by applying the coefficient vector for that pixel to the kernel dictionary. The neural network applies each denoising kernel to the respective pixel to generate a denoised output image.

Kernel prediction with kernel dictionary in image denoising

Growing community of inventors

St. Louis, MO, United States of America

Zhihao Xia