System and methods for performing nlp related tasks using contextualized word representations

Abstract

Systems, apparatuses, and methods for representing words or phrases, and using the representation to perform NLP and NLU tasks, where these tasks include sentiment analysis, question answering, and conference resolution. Embodiments introduce a type of deep contextualized word representation that models both complex characteristics of word use, and how these uses vary across linguistic contexts. The word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. These representations can be added to existing task models and significantly improve the state of the art across challenging NLP problems, including question answering, textual entailment and sentiment analysis.