Natural language processing techniques using multi-context self-attention machine learning frameworks

Abstract

Various embodiments of the present invention provide methods, apparatus, systems, computing devices, computing entities, and/or the like for performing natural language processing operations using a multi-context convolutional self-attention machine learning framework that comprises a shared token embedding machine learning model, a plurality of context-specific self-attention machine learning models, and a cross-context representation inference machine learning model, where each context-specific self-attention machine learning model is configured to generate, for each input text token of an input text sequence, a context-specific token representation using a context-specific self-attention mechanism that is associated with the respective distinct context window size for the context-specific self-attention machine learning model.