Baidu USA LLC

Jianwen Xie

Ping Li

Zilong Zheng

Yang Zhao

Different from prior works that model the internal distribution of patches within an image implicitly with a top-down latent variable model (e.g., generator), embodiments explicitly represent the statistical distribution within a single image by using an energy-based generative framework, where a pyramid of energy functions, each parameterized by a bottom-up deep neural network, are used to capture the distributions of patches at different resolutions. Also, embodiments of a coarse-to-fine sequential training and sampling strategy are presented to train the model efficiently. Besides learning to generate random samples from white noise, embodiments can learn in parallel with a self-supervised task (e.g., recover an input image from its corrupted version), which can further improve the descriptive power of the learned model. Embodiments does not require an auxiliary model (e.g., discriminator) to assist the training, and embodiments also unify internal statistics learning and image generation in a single framework.

Training energy-based models from a single image for internal learning and inference using trained models

Presented herein are embodiments of energy-based models (EBMs), which may be trained via embodiments of a multistage coarse-to-fine expanding and sampling strategy. Embodiments of the training methodology start with learning a coarse-level EBM from images at low resolution and then gradually transits to learn a finer-level EBM from images at higher resolution by expanding the energy function as the learning progresses. Embodiments are computationally efficient with smooth learning and sampling. Tested embodiments achieved the best performance on image generation amongst all EBMs and successfully synthesized high-fidelity images. Embodiments may also be used for image restoration and out-of-distribution detection. Framework embodiments may be further generalized for one-sided unsupervised image-to-image translation and beat baseline methods in terms of model size and training budget. Also presented herein are embodiments of a gradient-based generative saliency methodology to interpret the translation dynamics.

Growing community of inventors

Santa Clara, CA, United States of America

Jianwen Xie