Blackbox optimization via model ensembling

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for optimizing a plurality of parameters of a black box system with respect to a distribution of the plurality of parameters are disclosed. Methods can include obtaining a fitness function of the black box system and a training dataset including multiple training samples. Methods can further include training an ensemble of machine learning model wherein during each iteration of the training process an output of the system is generated for each of the multiple training samples. During each iteration, a predictive value is also generated using an ensemble of machine learning models. A natural gradient is determined based on the output of the system and the predictive value. The ensemble of machine learning models is then trained using the natural gradient.