Pre-screening training data for classifiers

Abstract

A system and method provide recommendations for refining training data that includes a training set of digital objects. A submitter labels the digital objects in the training set with labels, which may indicate whether the object is considered positive, neutral, or negative with respect to each of a predefined set of classes. Score vectors are computed by a trained categorizer for each digital object in the labeled training set. From the score vectors, various metrics are computed, such as a representative score vector and distances of score vectors from the representative score vector for a label group, cluster, or category of the categorizer. Based on the computed metrics, heuristics are applied and the training data is evaluated and recommendations may be made to the submitter, such as proposing that mislabeled objects are relabeled. The training data may include unlabeled digital objects, in which case, the recommendations may include suggestions for labeling the unlabeled objects.