Amazon Technologies, Inc.

Shanchan Wu

Arindam Mandal

Amitabh Saikia

Tagyoung Chung

Prakash Krishnan

Raefer Christopher Gabriel

Govindarajan Sundaram Thattai

Devesh Mohan Pandey

Aishwarya Naresh Reganti

Chien-wei Lin

Meyyappan Sundaram

Techniques for facilitating natural language interactions with visual interactive content are described. During a build time, a system analyzes various websites and applications relating to a particular user goal to understand website and application navigation and information relating to the user goal. The learned information is used to store configuration data. During runtime, when a user request performance of an action, the system engages in a dialog with the user to complete the user's goal. The system uses the stored configuration data to determine actions to be performed at a website or application to complete the user's goal, and determines system responses to present to the user to facilitate completion of the goal. Such system responses may request information from the user, may inform the user of information displayed at the website or application, etc.

Natural language interactions with interactive visual content

Growing community of inventors

Foster City, CA, United States of America

Shanchan Wu