Amazon Technologies, Inc.

Anushree Venkatesh

Raefer Christopher Gabriel

Arindam Mandal

Chandra Prakash Khatri

Behnam Hedayatnia

Ashish Kumar Agrawal

Ariya Rastrow

Nikko Strom

Ankur Gandhe

Ying Shi

Siddhartha Reddy Jonnalagadda

Angeliki Metallinou

Prakash Krishnan

Tagyoung Chung

Rahul Goel

Aaron Challenner

Shishir Sridhar Bharathi

Savas Parastatidis

Anirudh Raju

Veerdhawal Pande

Alexandros Potamianos

Vincent Auvray

Abhay Kumar Jha

David Chi-Wai Tang

Arijit Biswas

Xinyan Zhao

Nikolaos Malandrakis

Ahmad Shikib Mehri

Jiahui Li

Sanghyun Yi

Yun Lei

Arjit Biswas

Bonan Zheng

Yohan Jo

David A Jara

Techniques for determining when speech is directed at another individual of a dialog, and storing a representation of such user-directed speech for use as context when processing subsequently-received system-directed speech are described. A system receives audio data and/or video data and determines therefrom that speech in the audio data is user-directed. Based on this, the system determine whether the speech is able to be used to perform an action by the system. If the speech is able to be used to perform an action, the system stores a natural language representation of the speech. Thereafter, when the system receives system-directed speech, the system generates a rewrite of a natural language representation of the system-directed speech based on the previously-received user-directed speech. The system then determines output data responsive to the system-directed speech using the rewritten natural language representation.

Natural language generation

Techniques for storing and using multi-session context are described. A system may store context data corresponding to a first interaction, where the context data may include action data, entity data and a profile identifier for a user. Later the stored context data may be retrieved during a second interaction corresponding to the entity of the second interaction. The second interaction may take place at a system different than the first interaction. The system may generate a response during the second interaction using the stored context data of the prior interaction.

Multi-session context

A system that can engage in a dialog with a user may select a system response to a user input based on how the system estimates a user may respond to a potential system response. Models may be trained to evaluate a potential system response in view of various available data including dialog history, entity data, etc. Each model may score the potential system response for various qualitative aspects such as whether the response is likely to be comprehensible, on-topic, interesting, likely to lead to the dialog continuing, etc. Such scores may be combined to other scores such as whether the potential response is coherent or engaging. The models may be trained using previous dialog/chatbot evaluation data. At runtime the scores may be used to select a system response to a user input as part of the dialog.

Dialog response generation

Described herein is a system for rescoring automatic speech recognition hypotheses for conversational devices that have multi-turn dialogs with a user. The system leverages dialog context by incorporating data related to past user utterances and data related to the system generated response corresponding to the past user utterance. Incorporation of this data improves recognition of a particular user utterance within the dialog.

Speech recognition using dialog history

Growing community of inventors

San Mateo, CA, United States of America

Anushree Venkatesh