Energy-based generative models via coarse-to-fine expanding and sampling

Abstract

Presented herein are embodiments of energy-based models (EBMs), which may be trained via embodiments of a multistage coarse-to-fine expanding and sampling strategy. Embodiments of the training methodology start with learning a coarse-level EBM from images at low resolution and then gradually transits to learn a finer-level EBM from images at higher resolution by expanding the energy function as the learning progresses. Embodiments are computationally efficient with smooth learning and sampling. Tested embodiments achieved the best performance on image generation amongst all EBMs and successfully synthesized high-fidelity images. Embodiments may also be used for image restoration and out-of-distribution detection. Framework embodiments may be further generalized for one-sided unsupervised image-to-image translation and beat baseline methods in terms of model size and training budget. Also presented herein are embodiments of a gradient-based generative saliency methodology to interpret the translation dynamics.