Amazon Technologies, Inc.

Shiladitya Roy

Wei Huang

Sarthak Anand

Ravi Chikkanayakanahalli Mallikarjuniah

Priya Rao Chagaleti

Christopher Forbes Will

Cole Ira Brendel

Vikram Kumar Gundeti

Andrew Stuart Huntwork

Prashant J Thakare

Bo Li

Xu Zhang

Eswara Jnana Swaroop Bhupathiraju

Rohit Bhattacharjee

Nikhil Gupta

Ruhi Sarikaya

Rongzhou Shen

Clare Elizabeth Veladanda

Nicholas Adam Cummings

Alborz Geramifard

Monty Eich

Veer Yuganter Singh

Sabrina Chandrasekaran

Nathan G Leupold

Alekya Jonnala

Sol Jee Cho

David Z Chen

Pranav Kumar Singh

Techniques for resuming a skill session are described. A system receives a user input, determines a skill to execute with respect to the user input, opens a skill session, and causes the skill to execute. After the skill executes, the skill indicates the skill session is to be placed in a background state, resulting in the user being able to conduct one or more skill sessions while the skill processes in the background state. Sometime after placing the skill session in the background state, the skill requests the system to resume the skill session. If the system determines the skill session is to be resumed, the system sends data to the skill indicating that the skill session will be resumed. Once the skill session is to be resumed, the system places the skill session in an active state, and calls the skill to resume processing of the skill session.

Skill session backgrounding and resumption

A speech-processing system may provide access to one or more skills via spoken commands and/or responses in the form of synthesized speech. The system may be capable of keeping one or more skills active in the background while a user interacts (e.g., provides inputs to and/or receives outputs from) with a skill running in the foreground. A background skill may receive some trigger data, and determine to request the system to return the background skill to the foreground to, for example, request a user input regarding an action previously requested by the user. In some cases, the user may invoke a background skill to continue a previous interaction. The system may return the background skill to the foreground. The resumed skill may continue a previous interaction to, for example, to query the user for instructions, provide an update or alert, or continue a previous output.

Task resumption in a natural understanding system

Techniques for generating a personalization value that measures how tailored certain system interactions are for a user are described. A dialog exchange between a user and a skill may be determined, with the dialog exchange including user input data and system output data. It may be determined that the system output data was generated without respect to at least one previous user input or system output of the dialog exchanges. Based on this, a personalization value may be generated and sent to the skill.

Natural language dialog scoring

Described herein is a system for enabling a user to multitask by allowing a user to pause or interrupt an on-going interaction with a skill. The system monitors a state of a skill session, and updates the state to allow the user or system to suspend the session. The user may provide an instruction to pause an active session, causing the system to place the session in a suspended state. The user may then provide an instruction to resume the suspended session, causing the system to place the session in an active state. In other cases, the user input may be a request during an active session that requires invoking another skill. The system may place the current session in a suspended state, and invoke a second skill session to monitor the interaction with a second skill. When the interaction with the second skill is completed, the system may resume the previous session by placing it in an active state.

Multi-tasking and skills processing

The present disclosure provides various metrics that may be used to generate a score representing a conversational quality of a user/computing system dialog. User input data, corresponding to a dialog with a natural language processing system, may be determined and associated with a dialog identifier. System output data, corresponding to the dialog, may also be determined and associated with the dialog identifier. After the system output data is output to a user, the user input data and system output data may be processed to determine a score corresponding to the dialog exchange.

Techniques for generating a conformance score for a system/user dialog are described. A conformance score may represent a degree to which output data, provided by a skill, conforms to various policies (e.g., the data includes content appropriate for the age of the user, the data does not include profanity, etc.). User input data and system output data, corresponding to a dialog exchange between a user and a skill, ma be determined. A user type associated with the dialog exchange may also be determined. Based on the user type and the system output data, it may be determined that one or more filtering resources are to be assigned to process future data, received from the skill, prior to the future data being presented to a user.

Techniques for generating a prompt coverage score, which measures an extent to which data output to a user during a dialog is repetitive and monotonous, are described. User input data and system output, corresponding to a dialog exchange between a user and a skill, may be determined. A portion of the system output data, corresponding to a system prompt representing default output data, may be determined. A first number, representing possible variants of the prompt, may be determined along with a second number, representing variants of the prompt output during the dialog exchange. A prompt coverage score may be determined based on the first and second numbers.

Described herein is a system for allocating resources among multiple skills to enable multitasking. The system tracks use of resources using skill sessions. In one case, the system suspends a skill session to release a resource for allocation to another resource. In another case, the system determines if multiple skill sessions can remain active and use resources to provide output to the user at the same time.

Multi-tasking resource management

Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

Growing community of inventors

Bellevue, WA, United States of America

Shiladitya Roy