Amazon Technologies, Inc.

Chien-wei Lin

Arindam Mandal

Tagyoung Chung

Dilek Zeynep Hakkani-Tur

Amitabh Saikia

Angeliki Metallinou

Prakash Krishnan

Raefer Christopher Gabriel

Govindarajan Sundaram Thattai

Anish Acharya

Devesh Mohan Pandey

Aishwarya Naresh Reganti

Shachi Paul

Shubhra Chandra

Meyyappan Sundaram

Shanchan Wu

Techniques for facilitating natural language interactions with visual interactive content are described. During a build time, a system analyzes various websites and applications relating to a particular user goal to understand website and application navigation and information relating to the user goal. The learned information is used to store configuration data. During runtime, when a user request performance of an action, the system engages in a dialog with the user to complete the user's goal. The system uses the stored configuration data to determine actions to be performed at a website or application to complete the user's goal, and determines system responses to present to the user to facilitate completion of the goal. Such system responses may request information from the user, may inform the user of information displayed at the website or application, etc.

Natural language interactions with interactive visual content

A dialog generator receives data corresponding to desired dialog, such as application programming interface (API) information and sample dialog. A first model corresponding to an agent simulator and a second model corresponding to a user simulator take turns creating a plurality of dialog outlines of the desired dialog. The dialog generator may determine that one or more additional APIs are relevant to the dialog and may create further dialog outlines related thereto. The dialog outlines are converted to natural dialog to generate the dialog.

Goal-oriented dialog generation using dialog template, API, and entity data

Growing community of inventors

Sunnyvale, CA, United States of America

Chien-wei Lin