Efficient neural networks via ensembles and cascades

Abstract

A combination of two or more trained machine learning models can exhibit a combined accuracy greater than the accuracy of any one of the constituent models. However, this increase accuracy comes at additional computational cost. Cascades of machine learning models are provided herein that result in increased model accuracy and/or reduced model compute cost. These benefits are obtained by conditionally executing one or more of the models of the cascade based on the estimated correctness of already-executed models. The estimated correctness can be obtained as an additional output of the already-executed model(s) or could be determined as an entropy, maximum class probability, maximum class logit, or other function of the output(s) of the already-executed model(s). The expected computational cost of executing the model cascade is reduced by only executing the downstream model(s) when the upstream model(s) has resulted in an output whose accuracy is suspect.