Self-learning framework of zero-shot cross-lingual transfer with uncertainty estimation

Abstract

A method provided for cross-lingual transfer trains a pre-trained multi-lingual language model based on a gold labeled training set in a source language to obtain a trained model. The method assigns each sample in an unlabeled target language set to a silver label according to a model prediction by the trained model to obtain set of silver labels, and performs uncertainty-aware label selection based on the silver label assigned to each sample according to the model prediction and the trained model to obtain selected silver labels. The method performs iterative training on the selected labels by applying the selected silver labels in the target language set as training labels and re-training the trained model with the gold labels and the selected silver labels to obtain an iterative model, and performs task-specific result prediction in target languages based on the iterative model to generate a final predicted result in target languages.