Microsoft Technology Licensing, LLC

Samuel McIntire Thomson

Jason Michael Eisner

Benjamin David Van Durme

Eui Chul Shin

Emmanouil Antonios Platanios

Michael Jack Newman

Daniel Louis Klein

Adam David Pauls

Pengyu Chen

Hao Fang

Jiawei Zhou

Christopher H Lin

Subhro Roy

Harsh Jhamtani

Patrick Aozhe Xia

Kaj Alexander Nelson Bostrom

Benjamin Lev Snyder

Implementations of semantic parsing using pre-trained language models are provided. One aspect includes a computing system for semantic parsing of natural language. The computing system comprises processing circuitry and memory containing instructions that, when executed, cause the processing circuitry to receive a request comprising a natural language utterance and generate a formal meaning representation using the natural language utterance and a language model comprising a semantic parser that has been prompted with training data generated by providing a dataset comprising a set of unlabeled programmatic scripts and a seed programmatic script, generating a set of parsed natural language descriptions by inputting the set of unlabeled programmatic scripts into an inverse semantic parser, generating a set of re-parsed programmatic scripts by inputting the set of parsed natural language descriptions into the semantic parser, and determining a set of labeled programmatic scripts by validating the set of re-parsed programmatic scripts.

Semantic parsing with pre-trained language models

Systems and methods are provided for automatically generating a program based on a natural language utterance using semantic parsing. The semantic parsing includes translating a natural language utterance into instructions in a logical form for execution. The methods use a pre-trained natural language model and generate a canonical utterance as an intermediate form before generating the logical form. The natural language model may be an auto-regressive natural language model with a transformer to paraphrase a sequence of words or tokens in the natural language utterance. The methods generate a prompt including exemplar input/output pairs as a few-shot learning technique for the natural language model to predict words or tokens. The methods further use constrained decoding to determine a canonical utterance, iteratively selecting sequence of words as predicted by the model against rules for canonical utterances. The methods generate a program based on the canonical utterance for execution in an application.

Semantic parsing of utterance using contractive paraphrasing

A computer-implemented method is presented. The method comprises sequentially receiving a plurality of utterance prefixes, each sequentially received utterance prefix derived from a progressively longer incomplete portion of a full user utterance. For each sequentially received utterance prefix, a complete dataflow program is predicted based on the utterance prefix. The complete dataflow program includes a plurality of program nodes to be executed to satisfy the full user utterance. One or more program nodes are selected from the predicted complete dataflow program to speculatively execute based on at least the utterance prefix.

Speculative execution of dataflow program nodes

Growing community of inventors

Berkeley, CA, United States of America

Samuel McIntire Thomson