Multi-task learning for natural language processing tasks using a shared pre-trained language model

Abstract

Disclosed are machine learning techniques directed to training a machine learning model for the combined learning of multiple natural language processing (NLP) tasks. The NLP tasks may be named entity recognition (NER), relation extraction (RE), and assertion detection (AD) tasks. The machine learning model may be a multi-layer transformer model. Training the machine learning model may involve first training the NER module on the NER task, and thereafter training the RE module on the RE task while the AD module is simultaneously trained on the AD task. Training the machine learning model may alternatively involve training the NER module on the NER task concurrently with training the RE module on the RE task and training the AD module on the AD task. The trained machine learning model can predict entities and entity types in newly provided text, along with relations between the entities and assertions associated with the entities.