Diffusion-based image synthesis with defect blending via feature-level optimization

Abstract

A computer-implemented method generates a synthetic image with a blended defect, combining a first and second defect. A target feature map is created by merging masked feature maps derived from source images displaying the respective defects. These feature maps are generated using a machine learning model during a forward diffusion process. A noisy latent representation is obtained from an input image via another forward diffusion process. A new latent representation is then created through reverse diffusion, involving: (i) minimizing an energy function that compares a masked current feature map with the target feature map, (ii) predicting noise in the current iteration via the machine learning model, and (iii) updating the latent representation. The final synthetic image is produced by decoding the updated latent representation.