Amazon Technologies, Inc.

Devesh Mohan Pandey

Arindam Mandal

Prakash Krishnan

Raefer Christopher Gabriel

Amitabh Saikia

Tagyoung Chung

Govindarajan Sundaram Thattai

Aishwarya Naresh Reganti

Chien-wei Lin

Kjel Larsen

Meyyappan Sundaram

Shanchan Wu

Techniques for facilitating natural language interactions with visual interactive content are described. During a build time, a system analyzes various websites and applications relating to a particular user goal to understand website and application navigation and information relating to the user goal. The learned information is used to store configuration data. During runtime, when a user request performance of an action, the system engages in a dialog with the user to complete the user's goal. The system uses the stored configuration data to determine actions to be performed at a website or application to complete the user's goal, and determines system responses to present to the user to facilitate completion of the goal. Such system responses may request information from the user, may inform the user of information displayed at the website or application, etc.

Natural language interactions with interactive visual content

A dialog system receives a user request corresponding to a dialog with a user. The dialog system processes the user request to determine multiple service providers capable of responding to the user request. The dialog system selects one service provider based on a request-to-handle score, and selects another service provider based on a satisfaction rating. The dialog system updates the dialog state based on further input provided by the user to determine an output responsive to the user request.

Goal-oriented dialog system

Growing community of inventors

Redwood City, CA, United States of America

Devesh Mohan Pandey