Offline evaluation of machine learning models with noise reduction

Abstract

Methods, systems, and computer-readable media for offline evaluation of machine learning models with noise reduction are disclosed. A trigger computation system generates a plurality of experimental requests. The experimental requests do not represent unmodified requests received from clients in a production environment. At least one parameter value varies for individual ones of the experimental requests. The trigger computation system provides the experimental requests to a first machine learning model and a second machine learning model. The first machine learning model and the second machine learning model produce a set of results based at least in part on the experimental requests. The trigger computation system determines a reduced set of results for which the first machine learning model and the second machine learning model differ. An evaluation of the first machine learning model or the second machine learning model is performed using the reduced set of results.