Amazon Technologies, Inc.

Other

David Robert Thomas

Isaac Joseph Madwed

Julia Kennedy Nemer

Young-Bum Kim

Rajesh Kumar Pandey

Ruhi Sarikaya

Sumedha Arvind Kshirsagar

Ajay Gopalakrishnan

Nishant Kumar

Shubham Katiyar

Joe Pemberton

Jeff Bradley Beal

Vikas Jain

Andres Felipe Borja Jaramillo

Shishir Sridhar Bharathi

Alberto Milan Gutierrez

Rashmi Tonge

Mohamed Farouk AbdelHady

Arun Kumar Thenappan

Shijian Zheng

Shrish Chandra Mishra

Jihwan Lee

Adam D Baran

Kevindra Pal Singh

Rohin Dabas

Troy Dean Schuring

Michael James Montgomery

Nnenna Eleanya Okwara

Hao Yuan

Prasha Shrestha

Ricky C Pasch

Swaminathan Sivasubramanian

Nafea Bshara

Kurt W Piersol

Sudipta Sengupta

Vikram Kumar Gundeti

Leo Parker Dirac

Andrea Olgiati

Rahul Sharma

Nikko Strom

Chenlei Guo

Poorna Chand Srinivas Perumalla

Matthew James Wood

James David Meyers

Peter Spalding VanLund

Dominic Rajeev Divakaruni

Bratin Saha

Shuting Tang

Jalaja C Kurubarahalli

Sungjin Lee

Steven Mack Saunders

Pragya Agarwal

Andrew Christopher Miles

Rajankumar Singh

Zhenyu Yao

Umang Wadhwa

Wangyao Ge

Arima Vu Ram Thayumanavar

Pablo Cesar Ganga

Anna Caitlin Jentoft

Laura Maggia Panfili

Joo-Kyung Kim

Zdravko Pantic

Mark Klunder

Jacob Michael Simpson

Daniel Evans

Eftiquar Shaikh

Jun Wu

Cory Pruce

Samuel Oshin

Kurt Wesley Piersol

Techniques for routing a user command to a speechlet and resolving conflicts between potential speechlets are described. A system determines an intent of an input command. The system also receives context information associated with the input command. The system determines speechlets (e.g., speechlets and/or skills) that may execute with respect to the input command given the intent and the context data. The system then determines whether conditions of routing rules, associated with the speechlets, are satisfied given the context data. If the conditions of only one routing rule are satisfied, the system causes the speechlet associated with the routing rule to execute with respect to the input command. If the conditions of more than one routing rule are satisfied, the system may determine a speechlet to execute with respect to the input command based on the speechlets' priorities in a list of speechlets and/or based on potential output data provided by the speechlets.

System command processing

Described herein is a system for enabling a user to perform complex goals using multiple skills/applications of an intelligent assistant device. Skills may register as consumers of an action or providers of an action, and the consumer skills may be configured to invoke provider skills to perform actions. The system receives a request to perform an action from a skill along with some action data. The system validates the action data, selects another skill to perform the action, and forwards the request to the selected skill to perform the action.

Multiple skills processing

Techniques for intelligently selecting a component to execute with respect to a natural language user input are described. A natural language processing (NLP) system may receive first data representing a natural language input. The NLP system may determine first and second scores representing first and second confidences that first and second components are to be invoked to perform actions responsive to the natural language input, respectively. Based on the first and second scores, the NLP system may determine further information is needed to determine which of the first or second component is to be invoked. The NLP system may query a user for the further information. Based on the further information, the NLP system may determine third and fourth scores representing third and fourth confidences that the first and second components are to be invoked to perform actions responsive to the natural language input, respectively. The NLP system may determine the third score is greater than the fourth score and, based thereon, cause the first component to perform an action responsive to the original natural language input.

Natural language input processing

Features are disclosed for performing functions in response to user requests. Natural Language Understanding ('NLU') processing may be performed to generate command data that represents a subject of an utterance. The command data may be sent to an application that causes presentation of first output content in a first modality at a first time in response to receiving the command data, and generates second output content in a second modality different from the first modality, wherein the second output content is associated with the first output content. The second output content may be presented in the second modality at a second time subsequent to the first time.

Dialog management with multiple modalities

Devices and techniques are generally described for a speech processing routing architecture. In various examples, first data comprising a first feature definition is received. The first feature definition may include a first indication of first source data and first instructions for generating feature data using the first source data. In various examples, the feature data may be generated according to the first feature definition. In some examples, a speech processing system may receive a first request to process a first utterance. The feature data may be retrieved from a non-transitory computer-readable memory. The speech processing system may determine a first skill for processing the first utterance based at least in part on the feature data.

Natural language processing routing

Described herein is a system for responding to a frustrated user with a response determined based on spoken language understanding (SLU) processing of a user input. The system detects user frustration and responds to a repeated user input by confirming an action to be performed or presenting an alternative action, instead of performing the action responsive to the user input. The system also detects poor audio quality of the captured user input, and responds by requesting the user to repeat the user input. The system processes sentiment data and signal quality data to respond to user inputs.

Sentiment aware voice user interface

Techniques for partitioning data flow operations between execution on a compute instance and an attached accelerator instance are described. A set of operations supported by the accelerator is obtained. A set of operations associated with the data flow is obtained. An operation in the set of operations associated with the data flow is identified based on the set of operations supported by the accelerator. The accelerator executes the first operation.

Framework integration for instance-attachable accelerator

Devices and techniques are generally described for using user feedback to determine routing decisions in a speech processing system. In various examples, first data representing a first utterance may be received. Second data representing a first semantic interpretation of the first utterance may be determined. A first intent data processing application may be selected for processing the second data. Feedback data may be determined related to the first intent data processing application processing the second data. Third data representing a semantic interpretation of a second utterance may be received, wherein the first semantic interpretation is the same as the second semantic interpretation. A second intent data processing application may be determined for processing the third data based at least in part on the feedback data.

Natural language processing

Routing of natural language inputs to speech processing applications

Features are disclosed for performing functions in response to user requests based on contextual data regarding prior user requests. Users may engage in conversations with a computing device in order to initiate some function or obtain some information. A dialog manager may manage the conversations and store contextual data regarding one or more of the conversations. Processing and responding to subsequent conversations may benefit from the previously stored contextual data by, e.g., reducing the amount of information that a user must provide if the user has already provided the information in the context of a prior conversation. Additional information associated with performing functions responsive to user requests may be shared among applications, further improving efficiency and enhancing the user experience.

Dialog management with multiple applications

Dialog management context sharing

A speech-based system includes an audio device in a user premises and a network-based service that supports use of the audio device by multiple applications. The audio device may be directed to play audio content such as music, audio books, etc. The audio device may also be directed to interact with a user through speech. The network-based service monitors event messages received from the audio device to determine which of the multiple applications currently has speech focus. When receiving speech from a user, the service first offers the corresponding meaning to the application, if any, that currently has primary speech focus. If there is no application that currently has primary speech focus, or if the application having primary speech focus is not able to respond to the meaning, the service then offers the user meaning to the application that currently has secondary speech focus.

Application focus in speech-based systems

Growing community of inventors

Woodinville, WA, United States of America

David Robert Thomas