Data centric mislabel detection

Abstract

A system or method for identifying mislabels in a training dataset. The labeled dataset includes multiple labeled samples, each of which is labeled with a ground-truth label. The labeled samples are divided into multiple training subset and hold-out test subset. For each training subset and corresponding hold-out test subset, a machine learning model is trained using the training subset, and the trained machine learning model is applied to the hold-out test subset to generate prediction labels for samples in the corresponding hold-out test subset. Each prediction label has a confidence score indicating a likelihood of the prediction label is correct. A pair of prediction label and ground truth label corresponding to a same sample is compared to determine whether there is a candidate mislabel. The system determines whether the candidate mislabel is a mislabel based in part on a confidence score of the prediction label.