Microsoft Technology Licensing, LLC

Semantic Machines, Inc.

Daniel Louis Klein

David Leo Wright Hall

Jesse Daniel Eskes Rusak

Jordan Rian Cohen

Daniel Lawrence Roth

Adam David Pauls

Laurence Steven Gillick

Jacob Daniel Andreas

Pengyu Chen

Jason Andrew Wolfe

Percy Shuo Liang

David Ernesto Heekin Burkett

Michael Jack Newman

Jayant Sivarama Krishnamurthy

Andrew Volpe

Steven Andrew Wegmann

Taylor Darwin Berg-Kirkpatrick

Alexander J Kolmykov-Zotov

John P Bufe, Iii

Joshua James Clausman

Alan Xinyu Guo

Benjamin David Van Durme

Jason Michael Eisner

Samuel McIntire Thomson

Emmanouil Antonios Platanios

Linda Kathleen Arsenault

Eui Chul Shin

Christopher H Lin

Subhro Roy

Benjamin Lev Snyder

Jordan R Cohen

Systems and methods are provided for automatically generating a program based on a natural language utterance using semantic parsing. The semantic parsing includes translating a natural language utterance into instructions in a logical form for execution. The methods use a pre-trained natural language model and generate a canonical utterance as an intermediate form before generating the logical form. The natural language model may be an auto-regressive natural language model with a transformer to paraphrase a sequence of words or tokens in the natural language utterance. The methods generate a prompt including exemplar input/output pairs as a few-shot learning technique for the natural language model to predict words or tokens. The methods further use constrained decoding to determine a canonical utterance, iteratively selecting sequence of words as predicted by the model against rules for canonical utterances. The methods generate a program based on the canonical utterance for execution in an application.

Semantic parsing of utterance using contractive paraphrasing

A method includes receiving an utterance at a computerized automated assistant system, and detecting, via a date/time constraint module of the computerized automated assistant system, one or more constraints in the utterance associated with a date or time. The utterance is associated with a domain. The method further comprises generating, via the date/time constraint module, a periodic set for each of the one or more constraints associated with the date or time, and combining, via the date/time constraint module, the one or more periodic sets. The method further comprises processing, via a dialogue manager module of the computerized automated assistant system, the combined periodic sets to determine an action, and executing the action at the computerized automated assistant system.

Inference on date time constraint expressions

An automated natural dialogue system provides a combination of structure and flexibility to allow for ease of annotation of dialogues as well as learning and expanding the capabilities of the dialogue system based on natural language interactions.

Robust expandable dialogue system

A system that provides a sharable language interface for implementing automated assistants in new domains and applications. A dialogue assistant that is trained in a first domain can receive a specification in a second domain. The specification can include language structure data such as schemas, recognizers, resolvers, constraints and invariants, actions, language hints, generation template, and other data. The specification data is applied to the automated assistant to enable the automated assistant to provide interactive dialogue with a user in a second domain associated with the received specification. In some instances, portions of the specification may be automatically mapped to portions of the first domain. By having the ability to learn new domains and applications through receipt of objects and properties rather than retooling the interface entirely, the present system is much more efficient at learning how to provide interactive dialogue in new domains than previous systems.

Developer platform for providing automated assistant in new domains

A method comprising recognizing a user utterance including an ambiguity. The method further comprises using a previously-trained code-generation machine to produce, from the user utterance, a data-flow program including a search-history function. The search-history function is configured to select a highest-confidence disambiguating concept from one or more candidate concepts stored in a context-specific dialogue history.

Ambiguity resolution with dialogue search history

A method for configuring an automated dialogue system uses traces of interactions via a graphical user interface (GUI) for an application. Each trace includes interactions in the context of a plurality of presentations of the GUI. Elements of one or more presentations of the GUI are identified, and templates are associated with portions of the trace. Each template has one or more defined inputs and a defined output. For each template of the plurality of templates, the portions of the traces are processed to automatically configure the template by specifying a procedure for providing values of inputs to the template via the GUI and obtaining a value of an output. The automated dialogue system is configured with the configured templates, thereby avoiding manual configuration of the dialogue system.

Automatic spoken dialogue script discovery

A method for generating training data for training a natural language processing system comprises loading, into a computer memory, a computer-readable transcript representing an ordered sequence of one or more dialogue events. The method further comprises acquiring a computer-readable command describing an exemplary ordered subsequence of one or more dialogue events from the computer-readable transcript. The method further comprises re-parametrizing the computer-readable command with an alternative semantic parameter. The method further comprises generating an alternative ordered subsequence of one or more dialogue events based on the re-parametrized computer-readable command. The method further comprises outputting, to a data store, an alternative computer-readable transcript including the alternative ordered subsequence of one or more dialogue events, the alternative computer-readable transcript having a predetermined format usable to train the computerized assistant.

Training natural language system with generated dialogues

A method for generating a dialogue event in a natural language processing system comprises loading, into a computer memory, a computer-readable seed command describing an ordered sequence of two or more top-level dialogue events. A dialogue event includes a client utterance or a computerized assistant response. The seed command includes one or more sub-commands, each sub-command corresponding to a portion of the ordered sequence of two or more top-level dialogue events, and the focal sub-command of the one or more sub-commands being parametrized by a seed semantic parameter. The method further comprises re-parametrizing the focal sub-command by outputting a plurality of different re-parametrized focal sub-commands wherein, in each re-parametrized focal sub-command, the seed semantic parameter is replaced by one of a plurality of different synthetic semantic parameters. The method further comprises, for each of the plurality of different synthetic semantic parameters: saving a corresponding re-parametrized focal sub-command.

Generating dialogue events for natural language system

A method, comprising recognizing a user utterance for processing. The method further comprises using a previously-trained code-generation machine to generate, from the user utterance, a data-flow program configured to produce a return value upon successful execution. The method further comprises beginning execution of the data-flow program. Responsive to reaching an error condition resulting from execution of the data-flow program, the method further comprises, prior to the data-flow program producing the return value, suspending execution of the data flow program. The method further comprises using the previously-trained code-generation machine to generate an error-handling data-flow program, wherein the error-handling data-flow program is configured to produce the return value; beginning execution of the error-handling data-flow program to produce the return value; and outputting the return value.

Error recovery for conversational systems

A data collection system is based on a general set of dialogue acts which are derived from a database schema. Crowd workers perform two types of tasks: (i) identification of sensical dialogue paths and (ii) performing context-dependent paraphrasing of these dialogue paths into real dialogues. The end output of the system is a set of training examples of real dialogues which have been annotated with their logical forms. This data can be used to train all three components of the dialogue system: (i) the semantic parser for understanding context-dependent utterances, (ii) the dialogue policy for generating new dialogue acts given the current state, and (iii) the generation system for both deciding what to say and how to render it in natural language.

Data collection for a new conversational dialogue system

A method for a dialogue system includes establishing a dialogue session between an application executing on a server and a remote machine. The dialogue session includes one or more utterances received from a user at the remote machine. A natural language processing machine identifies a request associated with a computer-readable representation of an utterance. A dialogue expansion machine generates a plurality of alternative actions for responding to the request. A previously-trained machine learning confidence model assesses a confidence score for each alternative. If a highest confidence score for a top alternative does not satisfy a threshold, the plurality of alternatives including the top alternative are transmitted to a remote machine (which may be the same remote machine or a different remote machine) for review by a human reviewer. After the dialogue system and/or the human reviewer select an alternative, computer-readable instructions defining the selected alternative are executed.

Rapid deployment of dialogue system

A system that generates natural language content. The system generates and maintains a dialogue state representation having a process view, query view, and data view. The three-view dialogue state representation is continuously updated during discourse between an agent and a user, and rules can be automatically generated based on the discourse. Upon a content generation event, an object description can be generated based on the dialogue state representation. A string is then determined from the object description, using a hybrid approach of the automatically generated rules and other rules learned from annotation and other user input. The string is translated to text or speech and output by the agent. The present system also incorporates learning techniques, for example when ranking output and processing annotation templates.

Growing community of inventors

Orinda, CA, United States of America

Daniel Louis Klein