Augmented diffusion inversion using latent trajectory optimization

Abstract

Augmented Denoising Diffusion Implicit Models ('DDIMs') using a latent trajectory optimization process can be used for image generation and manipulation using text input and one or more source images to create an output image. Noise bias and textual bias inherent in the model representing the image and text input is corrected by correcting trajectories previously determined by the model at each step of a diffusion inversion process by iterating multiple starts the trajectories to find determine augmented trajectories that minimizes loss at each step. The trajectories can be used to determine an augmented noise vector, enabling use of an augmented DDIM and resulting in more accurate, stable, and responsive text-based image manipulation.