Resource-efficient training of a sequence-tagging model

Abstract

A technique iteratively updates model weights of a teacher model and a student model. In operation, the teacher model produces noisy original pseudo-labeled training examples from unlabeled training examples. The technique weights the original pseudo-labeled training examples based on validation information. The technique then updates model weights of the student model based on the weighted pseudo-labeled training examples. The validation information, which is used to weight the original pseudo-labeled training examples, is produced by selecting labeled training examples based on an uncertainty-based factor and a similarity-based factor. The uncertainty-based factor describes an extent to which the student model produces uncertain classification results for the set of labeled training examples. The similarity-based factor describes the similarity between the set of labeled training examples and the unlabeled training examples. Overall, the technique is efficient because it eliminates the need to produce a large number labeled training examples.