Unified sample reweighting framework for learning with noisy data and for learning difficult examples or groups

Abstract

A method includes receiving training data for a machine learning model, the training data comprising a plurality of training examples and a corresponding plurality of labels. The method further includes dividing the training data into a plurality of training batches. For each training batch of the plurality of training batches, the method additionally includes learning a weight for each training example in the training batch that minimizes a sum of weighted losses for the training batch subject to a divergence constraint, where the divergence constraint limits a divergence of the learned weights for the training batch from a reference distribution, where the divergence is determined according to a chosen divergence measure. The method also includes training the machine learning model with each training batch of the plurality of training batches using the learned weight for each training example in the training batch. The method additionally includes providing the trained machine learning model.