Systems for context-aware named entity recognition

Abstract

To perform a Named Entity Recognition (NER) operation that accounts for semantic relationships between entity and document text while remaining scalable, entity text and text from portions of documents are encoded separately, in some cases using different encoders such as when the document text also includes layout information. A cross-attention module is then used to determine a weighted embedding for each document embedding, based on the document embedding and each entity embedding. The weighted embeddings include weights indicative of semantic relationships between the document text and the text of each entity. Correspondence between the weighted embeddings and the corresponding document embedding is then used to determine scores representing the probability that an entity is referenced in a portion of a document. Because the weighted embeddings are used, these scores account for semantic relationships between entity and document text.