Multi-stage training of a tagging model with query supplementation and adversarial enhancement

Abstract

A technique is described herein for training a tagging model that is able to successfully interpret queries. The technique trains the tagging model in plural stages. A first stage continues training a pre-trained language model based on a set of queries, to produce a first-stage model. A second stage performs training on the basis of a set of supplemented queries and associated weak labels, to produce a second-stage model. Each supplemented query combines a query with titles of documents that match the query. A third stage performs training on the basis of a set of supplemented queries and associated strong labels, to produce a third-stage model. The third stage also uses adversarial knowledge enhancement that has the effect of making the data presented to the third-stage model more difficult for the third-stage model to interpret. This, in turn, improves the generalization capabilities and robustness of the third-stage model.