Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Vasiliy Radostev

Ruhi Sarikaya

Anjishnu Kumar

Abhinav Sethy

Chetan Nagaraj Naik

Rekha Seshadrinathan

Spyridon Matsoukas

Nikko Strom

Chenlei Guo

Omar Zia Khan

Xing Fan

Jean-Philippe Robichaud

Alexandre Rochette

Paul Anthony Crook

Vipul Agarwal

Marius Alexandru Marin

Casey Stuart Smith

Xiaojiang Huang

Yi Luo

Jie Bao

Muddu Krishna Chintha

Priti Bisaria

A system may be configured to receive and process various signals to generate a natural language description of a user's environment, called situational context data. The signals may include sensor data, device status, user activity, user input, and/or inferences made using such data. The situational context data may express a user-centric description of the user's environment; for example: 'User is taking a walk in the park on a sunny afternoon' or “activity: driving location: highway”, etc. The system may send the situational context data to various system components that may, for example, process speech, select applications/skills for handling user inputs, and/or that implement those applications/skills. The applications/skills may use the situational context data to provide recommendations, generate responses, and/or perform actions that are more relevant to the user's current environment.

Determining device context

Techniques for generating a visual response to a user input are described. A system may receive a natural language input and use a machine learning model to determine a first component is to determine a response to the natural language input while a second component is to determine supplemental content related to the natural language input. The system may receive, from the first component, first image data corresponding to the response. The system may also receive, from the second component, second image data corresponding to the supplemental content. The system may send, to a display, a command to present the first image data and the second image data.

Visual responses to user inputs

Techniques for generating a visual response to a user input are described. A system may receive input data corresponding to a user input, determining a first skill component is to determine a response to the user input, and determine a second skill component is to determine supplemental content related to the user input. The system may also determine a template for presenting a visual response to the user input, where the template is configured for presenting the response and the supplemental content. The system may receive, from the first skill component, first image data corresponding to the first response. The system may also receive, from the second skill component, second image data corresponding to the first supplemental content. The system may send, to a device including a display, a command to present the first image data and the second image data using the template.

Conversational understanding systems allow users to conversationally interface with a computing device. In examples, a query may be received that includes a request for execution of a task. A data exchange task definition may be accessed. The data exchange task definition assists a conversational understanding system in managing task state tracking for information needed for task execution. Using the data exchange task definition, a per-turn policy for interacting with the user computing device is generated based on the state of a dialogue with a computing device and an evaluation of a process flow chart provided by a task owner resource. The task owner resource may be independent from the conversational understanding system. A response to the query may be generated and output based on the per-turn policy. In examples, the per-turn policy is used to generate one or more responses during a dialogue with a user via a computing device.

Policy authoring for task state tracking during dialogue

Aspects herein provide third-party application authors with a resolver chaining platform that simplifies the task of creating customized resolvers to gather information from user input while also allowing those authors to chain their custom resolvers with generic resolvers provided by the platform's host and that relate to commonly used parameter types. In particular, the present disclosure provides a method and system for authoring and using these resolver chains made up of a combination of custom and generic resolvers.

Framework for language understanding systems that maximizes reuse through chained resolvers

Systems and methods for augmenting existing CU system to be used with content, such as a website. The content may be parsed to determine on or more actions that may be performed by a user who uses the content. These actions may then be compared to tasks of CU systems to identify potential matches. When a match is found, the CU system may be updated to include information.

Automating natural language task/dialog authoring by leveraging existing content

Growing community of inventors

Seattle, WA, United States of America

Vasiliy Radostev