Microsoft Technology Licensing, LLC

Meta Platforms, Inc.

The Allen Institute for Artificial Intelligence

Luke Zettlemoyer

Christopher Brian Quirk

Christopher T Clark

Pallavi Choudhury

Armen Aghajanyan

Jurij Ganitkevic

Matthew E Peters

Kenton Lee

Mark Neumann

Sida Wang

Gargi Ghosh

Mohit Iyyer

Marjan Ghazvini Nejad

Matt Gardner

Michael William Lewis

In one embodiment, a method includes accessing a first document, accessing a plurality of second documents, calculating a relevance score for each of the plurality of second documents indicating a degree of relevance of the second document to the first document using an encoder of a machine-learning model, selecting a subset of the second documents based on their corresponding relevance scores, generating a target document by using the machine-learning model to process the subset of second documents and their corresponding relevance scores, and updating parameters of the machine-learning model based on a comparison between the first document and the generated target document.

Pretraining a language machine-learning model

Systems, apparatuses, and methods for representing words or phrases, and using the representation to perform NLP and NLU tasks, where these tasks include sentiment analysis, question answering, and conference resolution. Embodiments introduce a type of deep contextualized word representation that models both complex characteristics of word use, and how these uses vary across linguistic contexts. The word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. These representations can be added to existing task models and significantly improve the state of the art across challenging NLP problems, including question answering, textual entailment and sentiment analysis.

System and methods for performing NLP related tasks using contextualized word representations

A processing system is described which induces a context free grammar (CFG) based on a set of descriptions. The descriptions pertain to a particular subject. Thus, the CFG targets the particular subject, and is accordingly referred to as a subject-targeted context free grammar (ST-CFG). The processing system can use the ST-CFG to determine whether a new description is a proper description of the subject. The processing system also provides synthesizing functionality for building an ST-CFG based on one or more smaller component ST-CFGs.

Inducing and applying a subject-targeted context free grammar

Growing community of inventors

Seattle, WA, United States of America

Luke Zettlemoyer