Amazon Technologies, Inc.

Raytheon Bbn Technologies Corp.

Bbn Technologies Corporation

Rohit Prasad

Ariya Rastrow

Shiv Naga Prasad Vitaladevuni

Spyridon Matsoukas

Nikko Strom

Premkumar Natarajan

Bjorn Hoffmeister

Kerry Hammil

Ruhi Sarikaya

Arindam Mandal

Stephen Frederick Potter

Pradeep Natarajan

Christopher Schindler

Michael Martin George

William Folwell Barton

Kenneth John Basye

Young-Bum Kim

Rajiv Ramachandran

Anna Chen Santos

Joseph Pedro Tavares

Frédéric Johan Georges Deramat

Arushan Rajasekaram

Aparna Khare

Sundararajan Srinivasan

Shiv N Vitaladevuni

David Sanchez

Jared Strawderman

Sarah Castle

Jeffrey P Bezos

Dilek Zeynep Hakkani-Tur

Gregory Michael Hart

Brian Alexander Oliver

Jae Pum Park

Tony Roy Hardie

Yuzo Watanabe

Avnish Sikka

Timothy D Twerdahl

Madan Mohan Rao Jampani

Angeliki Metallinou

Andrew Smith

Arpit Gupta

Krishna Subramanian

Karthik Ramakrishnan

Abhinav Sethy

Savas Parastatidis

Rafal Kuklinski

Nikko Ström

Maria Christine Renz

Dean Curtis

Pushpendre Rastogi

Nitin Ashok Jain

John Arland Nave

Mayank Kumar

Nakul Dahiwade

Arshdeep Singh

Nikhil Reddy Kortha

Bartek Gulik

Gokhan Tur

Amit Singh Chhetri

Ohil Krishnamurthy Manyam

Ian W Freed

Manoj Sindhwani

Imre Attila Kiss

Ankur Gandhe

Hung The Pham

Siddhartha Reddy Jonnalagadda

Prakash Krishnan

Sri Venkata Surya Siva Rama Krishna Garimella

Denis Sergeyevich Filimonov

Tagyoung Chung

Markus Dreyer

Janet Louise Slifka

Raefer Christopher Gabriel

Shuang Wu

Joel Joseph Chengottusseriyil

Suranjit Adhikari

Simon Peter Reavely

Govindarajan Sundaram Thattai

Rajiv Reddy

Qing Ping

Feiyang Niu

Sai Sailesh Kopuri

Vincent Auvray

Qiaozi Gao

Roland Maas

Rohit Kumar

Matthew E Roy

Qing Liu

Sankaranarayanan Ananthakrishnan

Aishwarya Naresh Reganti

Minmin Shen

Hung Tuan Pham

Thomas Edward Taylor

Sridhar Yadav Manoharan

Ankita De

Julian Chan

Sanjika Hewavitharana

Frederick Choi

Phillip Ryan Hilmes

Described are systems and techniques configured to acquire data from user speech generated by one user of a materials handling facility speaking to themselves or another user. The user speech is processed using natural language processing to determine key words, concepts, meanings, and so forth. Based on the processed data, queries may be executed, data may be presented in user interfaces, operation of the materials handling facility may be modified, and so forth.

Verbally interactive materials handling facility

A system that detects the location of a user gaze at a display and in response to the duration of the gaze exceeding a threshold, auto-playing content on the display. The system may also determine gaze event data associating the gaze event with the source of the content the user is gazing at. Other information may also be associated with the gaze event such as user ID, time/duration data, or the like. Various actions can be taken in response to the gaze event such as auto-playing of content, outputting a visual indication of the detected gaze, interpreting detected speech using the gaze event data, data aggregation, etc.

Gaze initiated actions

Systems, methods, and computer-readable media are disclosed for dynamic voice search transitioning. Example methods may include receiving, by a computer system in communication with a display, a first incoming voice data indication, initiating a first user interface theme for presentation at a display, wherein the first user interface theme is a default user interface theme, and receiving first voice data. Example methods may include sending the first voice data to a remote server for processing, receiving an indication from the remote server to initiate a second user interface theme, and initiating the second user interface theme for presentation at the display.

Dynamic voice search transitioning

A natural language system may be configured to act as a participant in a conversation between two users. The system may determine when a user expression such as speech, a gesture, or the like is directed from one user to the other. The system may processing input data related the expression (such as audio data, input data, language processing result data, conversation context data, etc.) to determine if the system should interject a response to the user-to-user expression. If so, the system may process the input data to determine a response and output it. The system may track that response as part of the data related to the ongoing conversation.

Dialog management for multiple users

Techniques for recommending a skill experience to a user after a user-system dialog session has ended are described. Upon a dialog session ending, the system uses a first machine learning model to determine potential intents to recommend to a user. The system then uses a second machine learning model to determine a particular skill and intent to recommend. The system then prompts the user to accept the recommended skill and intent. If the user accepts, the system calls the recommended skill to execute. As part of calling the skill, the system sends to the skill at least one entity provided in a natural language user input of the ended dialog session. This enables the skill to skip welcome prompts, and initiate processing to output a response based on the intent and the at least one entity of the ended dialog session.

User-system dialog expansion

Devices and techniques are generally described for learning personalized concepts for natural language processing. In various examples, a first natural language input may be received. In some examples, a determination may be made that the first natural language input comprises non-actionable slot data. A dialog session may be initiated with the user. In some examples, first slot data that is indicated by the user during the dialog session may be determined. In various examples, data representing the first slot data may be stored in a database in association with the first natural language input.

Natural language processing

Techniques for enrolling a user in a system's user recognition functionality without requiring the user speak particular speech are described. The system may determine characteristics unique to a user input. The system may generate an implicit voice profile from user inputs having similar characteristics. After an implicit voice profile is generated, the system may receive a user input having speech characteristics similar to that of the implicit voice profile. The system may ask the user if the user wants the system to associate the implicit voice profile with a particular user identifier. If the user responds affirmatively, the system may request an identifier of a user profile (e.g., a user name). In response to receiving the user's name, the system may identify a user profile associated with the name and associate the implicit voice profile with the user profile, thereby converting the implicit voice profile into an explicit voice profile.

Speech based user recognition

Described are techniques for predicting when data associated with a user input is likely to be selected for deletion. The system may use a trained model to assist with such predictions. The trained model can be configured based on deletions associated with a user profile. An example process can including receiving user input data corresponding to the user profile, and processing the user input data to determine a user command. Based on characteristic data of the user command, the trained model can be used to determine that data corresponding to the user command is likely to be selected for deletion. The trained model can be iteratively updated based on additional user commands, including previously received user commands to delete user input data.

Predictive deletion of user input

A speech interface device is configured to receive response data from a remote speech processing system for responding to user speech. This response data may be enhanced with information such as remote NLU data. The response data from the remote speech processing system may be compared to local NLU data to improve a speech processing model on the device. Thus, the device may perform supervised on-device learning based on the remote NLU data. The device may determine differences between the updated speech processing model and an original speech processing model received from the remote system and may send data indicating these differences to the remote system. The remote system may aggregate data received from a plurality of devices and may generate an improved speech processing model.

On-device learning in a hybrid speech processing system

Features are disclosed for detecting words in audio using contextual information in addition to automatic speech recognition results. A detection model can be generated and used to determine whether a particular word, such as a keyword or 'wake word,' has been uttered. The detection model can operate on features derived from an audio signal, contextual information associated with generation of the audio signal, and the like. In some embodiments, the detection model can be customized for particular users or groups of users based usage patterns associated with the users.

Wake word detection modeling

Techniques for ensuring content output to a user conforms to a quality of the user's speech, even when a speechlet or skill ignores the speech's quality, are described. When a system receives speech, the system determines an indicator of the speech's quality (e.g., whispered, shouted, fast, slow, etc.) and persists the indicator in memory. When the system receives output content from a speechlet or skill, the system checks whether the output content is in conformity with the speech quality indicator. If the content conforms to the speech quality indicator, the system may cause the content to be output to the user without further manipulation. But, if the content does not conform to the speech quality indicator, the system may manipulate the content to render it in conformity with the speech quality indicator and output the manipulated content to the user.

Content output management based on speech quality

Systems, methods, and devices for computer-generating responses and sending responses to communications when the recipient of the communication is unavailable are disclosed. An individual may send a message (either audio or text) to a recipient. The recipient may be unavailable to contemporaneously respond to the message (e.g., the recipient may be performing an action that makes is difficult or impractical for the recipient to contemporaneously respond to the audio message). When the recipient is unavailable, a response to the message is generated and sent without receiving an instruction from the recipient to do so. The response may be sent to the message originating individual, and content of the response may thereafter be sent to the recipient to receive feedback regarding the correctness of the response. Alternatively, the response content may first be sent to the recipient to receive the feedback, and thereafter the response may be sent to the message originating individual.

Generation of automated message responses

Techniques for limiting natural language processing performed on input data are described. A system receives input data from a device. The input data corresponds to a command to be executed by the system. The system determines applications likely configured to execute the command. The system performs named entity recognition and intent classification with respect to only the applications likely configured to execute the command.

Natural language speech processing application selection

Growing community of inventors

Acton, MA, United States of America

Rohit Prasad