Training natural language processing models using modified cross entropy loss functions

Abstract

Methods of training a natural language processing (NLP) model comprise storing classification data associated with a plurality of words, defining model input data associated with each of the plurality of words, evaluating a cross entropy loss function to derive a loss value, and adjusting the NLP model according to the loss value derived from the cross entropy loss function. The cross entropy loss function may be defined to include one or more special loss terms to support training of the NLP model on the basis of incomplete classification data, such as in cases where not all possible classes are considered by the annotator, where not all words are provided with annotated classes, where consensus has not been reached between annotators, and/or where not all classes have been defined.