Amazon Technologies, Inc.

Premkumar Natarajan

Arindam Mandal

Yue Wu

Anoop Kumar

Jing Huang

Ayush Jaiswal

Jiao Sun

Shereen Oraby

Anjali Yuan Narayan-Chen

Dilek Zeynep Hakkani-Tur

Gokhan Tur

Rahul Gupta

Rohit Prasad

Yang Liu

Spyridon Matsoukas

Pradeep Natarajan

Tagyoung Chung

Vivek Yadav

Karthik Ramakrishnan

Melanie C B Gens

Joel Joseph Chengottusseriyil

Sankaranarayanan Ananthakrishnan

Anil Ramakrishna

Rajiv Reddy

Govindarajan Sundaram Thattai

Feiyang Niu

Qing Ping

Sriram Venkatapathy

Qiaozi Gao

Aishwarya Naresh Reganti

Aram Galstyan

Nanyun Peng

Alessandra Cervone

Heng Ji

Yi Fung

Nanyung Peng

Shuyang Gao

Prateek Singhal

Seyed Omid Razavi

Srinivasan Rengarajan Veeravanallur

Avni Khatri

Jwala Dhamala

Apurv Verma

Qingwen Ye

Mayur Himmatbhai Dabhi

Gourav Datta

Prashanth Rajagopal

In accordance with one disclosed method, first data representing first text may be processed using a first machine learning model configured to generate second data representing an explanation as to why the first text is humorous. The second data may be processed, together with the first data, by a second machine learning model configured to generate a value indicating that the first data and the second data correspond to a pun. A device may be caused to take at least a first action based at least in part on the value.

Virtual assistant humor management

Techniques for generating a natural language prompt to further a goal of a dialog, are described. During a dialog, the system receives one or more user inputs including a user question, a user response to the question, and a request to generate a further question following the response. The system determines ASR output data corresponding to the user inputs, and determines dialog history data of the dialog. Using the ASR output data and the dialog history data, the system determines a category and an explanation of relevance corresponding to the category. Using the ASR output data, the dialog history, the category, and the explanation, the system determines the further question to be output to the user.

Natural language question generation

In accordance with one disclosed method, first data representing one or more first keywords may be processed by at least one first component to determine a first pair of words suitable for generating a pun. The first pair of words may then be processed using a first machine learning model configured to generate an output pun based at least in part on one or more input keywords and an input word pair such that the output pun is contextually related to the one or more input keywords. A first pun that is contextually related to the one or more first keywords may be received from the first machine learning model, and a first device may be caused to output a representation of the first pun.

A system configured to perform style-aware listener animation. By representing different listening styles (e.g., facial expressions) using an embedding space, a single model can be trained to generate unique facial animations for a number of distinct listeners. Thus, individual listening styles can be associated with a listener identifier, enabling the system to (i) animate a plurality of different listeners with unique nonverbal behavior and/or (ii) select a particular listener identifier or desired type of listener style with which to animate. This enables the model to be generalized to new listeners to generate additional listener facial responses without needing training data for each new listener. The model may process a listener representation style or listener identifier, along with input data corresponding to a speaker talking, to generate unique facial animation responsive to the speech.

Listener animation

Devices and techniques are generally described for cohort determination in natural language processing. In various examples, a first natural language input to a natural language processing system may be determined. The first natural language input may be associated with a first account identifier. A first machine learning model may determine first data representing one or more words of the first natural language input. A second machine learning model may determine second data representing one or more acoustic characteristics of the first natural language input. Third data may be determined, the third data including a predicted performance for processing the first natural language input by the natural language processing system. The third data may be determined based on the first data representation and the second data representation.

Cohort determination in natural language processing

Devices and techniques are generally described for learning personalized concepts for natural language processing. In various examples, a first natural language input may be received. In some examples, a determination may be made that the first natural language input comprises non-actionable slot data. A dialog session may be initiated with the user. In some examples, first slot data that is indicated by the user during the dialog session may be determined. In various examples, data representing the first slot data may be stored in a database in association with the first natural language input.

Natural language processing

Devices and techniques are generally described for class-agnostic object detection. In some examples, a first frame of image data comprising a first plurality of pixels may be received. First class-agnostic feature data representing the first plurality of pixels may be generated. A first object detection component may be used to determine that the first plurality of pixels corresponds to an arbitrary object represented in the first frame of image data based at least in part on the first class-agnostic feature data. Class-agnostic data indicating that the first plurality of pixels in the first frame of image data corresponds to the arbitrary object may be generated.

Class-agnostic object detection

Techniques for using a federated learning framework to update machine learning models for spoken language understanding (SLU) system are described. The system determines which labeled data is needed to update the models based on the models generating an undesired response to an input. The system identifies users to solicit labeled data from, and sends a request to a user device to speak an input. The device generates labeled data using the spoken input, and updates the on-device models using the spoken input and the labeled data. The updated model data is provided to the system to enable the system to update the system-level (global) models.

Spoken language understanding models

Growing community of inventors

Rolling Hills Estates, CA, United States of America

Premkumar Natarajan