Automated variational inference using stochastic models with irregular beliefs

Abstract

A system and method for automated construction of a stochastic deep neural network (DNN) architecture is provided. The framework of invention automatically searches for most relevant stochastic modes underlaying datasets for variational Bayesian inference. The invention provides a way to use heterogenous, irregular, and mismatched beliefs in stochastic sampling for intermediate representation in DNNs with a capability of an automatically tuning mechanism of posterior, prior, and likelihood models to enable accurate generative models and uncertainty models for machine learning tasks. The system further allows adjustable discrepancy measure to regularize intermediate representation by variants of divergence metrics including Renyi's alpha, beta, and gamma divergences. The invention enables diverse mixture combinations of stochastic models for misspecified and unspecified probabilistic relations in an automatic fashion. Accordingly, the representation capability of variational autoencoders, variational information bottlenecks, denoising diffusion probabilistic models and other stochastic DNNs are improved.