Amazon Technologies, Inc.

Behavioral Signal Technologies, Inc.

Alexandros Potamianos

Chao Wang

Nikko Strom

Viktor Rozgic

Ying Shi

Siddhartha Reddy Jonnalagadda

Angeliki Metallinou

Aaron Challenner

Katherine M Ryan

Anushree Venkatesh

Veerdhawal Pande

Vincent Auvray

Avani Parakh

Arijit Biswas

David A Jara

Xinyan Zhao

Nikolaos Malandrakis

Jiahui Li

Yohan Jo

Bonan Zheng

James Gibson

Shrikanth Narayanan

Efthymis Georgiou

Georgios Paraskevopoulos

The present application provides techniques for implementing a skill component, configured to perform an assessment of a user, as part of a speech processing system. The system may receive a natural language user input requesting assistance. The skill component may, using one or more machine learning models, determine at least one characteristic of the natural language input (e.g., lexical embedding, acoustic embedding, topic, tone, etc.). The skill component may determine state data for a present session, where the state data indicates a topic of the natural language user input and/or a user state associated with the natural language user input. The skill component may determine past state data of one or more past sessions, and generate a question to the user based on the state data for the natural language user input and the past state data.

Conversation-based skill component for assessing a user's state

Techniques for determining when speech is directed at another individual of a dialog, and storing a representation of such user-directed speech for use as context when processing subsequently-received system-directed speech are described. A system receives audio data and/or video data and determines therefrom that speech in the audio data is user-directed. Based on this, the system determine whether the speech is able to be used to perform an action by the system. If the speech is able to be used to perform an action, the system stores a natural language representation of the speech. Thereafter, when the system receives system-directed speech, the system generates a rewrite of a natural language representation of the system-directed speech based on the previously-received user-directed speech. The system then determines output data responsive to the system-directed speech using the rewritten natural language representation.

Natural language generation

A method for processing multi-modal input includes receiving multiple signal inputs, each signal input having a corresponding input mode. Each signal input is processed in a series of mode-specific processing stages. Each successive mode-specific stage is associated with a successively longer scale of analysis of the signal input. A fused output is generated based on the output of a series of fused processing stages. Each successive fused processing stage is associated with a successively longer scale of analysis of the signal input. Multiple fused processing stages receive inputs from corresponding mode-specific processing stages, so that the fused output depends on the multiple of signal inputs.

Deep hierarchical fusion for machine intelligence applications

Growing community of inventors

Santa Monica, CA, United States of America

Alexandros Potamianos