Quantifying and improving the performance of computation-based classifiers

Abstract

Enhanced methods for improving the performance of classifiers are described. A ground-truth labeled dataset is accessed. A classifier predicts a predicted label for datapoints of the dataset. A confusion matrix for the dataset and classifier is generated. A credibility interval is determined for a performance metric for each label. A first labels with a sufficiently large credibility interval is identified. A second label is identified, where the classifier is likely to confuse, in its predictions, the first label with the second label. The identification of the second label is based on instances of incorrect label predictions of the classifier for the first and/or the second labels. The classifier is updated based on a new third label that includes an aggregation of the first label and the second label. The updated classifier model predicts the third label for any datapoint that the classifier previously predicted the first or second labels.