Amazon Technologies, Inc.

Wenbo Yan

Prathap Ramachandra

Liu Yang

Charlotte Alizerine Dzialo

Jingqian Zhao

Ponnu Jacob

Uday Kumar Kollu

Kyung Jae Lee

Lan Ma

Ruiqi Luo

Krupal Maddipati

Jinning Wu

Darshan Ashok Fofadiya

Yi Qin

Xi Chelsea Chen

Nalledath P Vinodkrishnan

Adriano Devillaine

Adam Kalman

Mohammad Alnuaimat

Daksh Gautam

Shu Wang

Deshen Wang

Di Wang

Meng Xie

Approaches are generally described for corrupted speech detection in voice-based computer interfaces. First input data including first audio data representing a user utterance may be received. First data representing the first audio data may be generated using a first encoder. First text data representing a transcription of the user utterance may be generated. Second data representing the first text data may be generated using a second encoder different from the first encoder. Third data may be generated by combining the first data and the second data. The third data may be sent to a classifier network trained to predict a relevant corruption state for speech processing inputs. The classifier network may determine that the first input data corresponds to a first corruption state.

Detecting corrupted speech in voice-based computer interfaces

Techniques are described and relate to providing computing services using embeddings of a transformer-based encoder. In an example, a computer system generates, by using a machine learning (ML) transformer, an embedding vector based at least in part on text. The computer system stores the embedding vector and an association between the embedding vector and the text in a data store. Further, the computer system determines that a task is to be performed based at least in part on natural language understanding (NLU) of the text. The computer system receives the embedding vector from the data store based at least in part on the association between the embedding vector and the text. The task is performed based at least in part on the embedding vector after being received from the data store.

Computing services using embeddings of a transformer-based encoder

A multi-tier domain is provided for processing user voice queries and making routing decisions for generating responses, including for user voice queries that include multi-domain trigger words or phrases. When an utterance is recognized as different intents in different domains, a routing system for a domain may consider contextual signals, including those associated with other domains, to determine whether the domain is the proper one to handle the request. This determination can be performed with a statistical model specifically trained to make such determinations using the available contextual data.

Multi-domain intent handling with cross-domain contextual signals

Features are disclosed for training and using a word embedding model configured to receive textual and context data associated with an utterance of a user. A word embedding model can be trained with text data and context data to account for context associated with the text data. The word embedding model can receive an input vector including text data and one or more sets of context data associated with the text data and perform word embedding based on the input vector. In some embodiments, the input vector can include an automatic speech recognition ('ASR') confidence score generated by an ASR model and one or more labels generated by an NLU model. In some embodiments, the input vector can include user information associated with the user.

Architecture for context-augmented word embedding

A speech processing system uses contextual data to determine the specific domains, subdomains, and applications appropriate for taking action in response to spoken commands and other utterances. Some applications may be given priority over others such that some applications are general request applications to which responsibility for processing an intent is to be assigned as long as contextual criteria are satisfied, while other applications are specific request applications to which responsibility for processing an intent is to be assigned only if the applications are specifically requested, if the contextual criteria of priority applications are not satisfied, and/or if certain contextual criteria associated with the specific request applications are satisfied.

Priority and context-based routing of speech processing

A speech processing system uses contextual data to determine the specific domains, subdomains, and applications appropriate for taking action in response to spoken commands and other utterances. The system can use signals and other contextual data associated with an utterance, such as location signals, content catalog data, data regarding historical usage patterns, data regarding content visually presented on a display screen of a computing device when an utterance was made, other data, or some combination thereof.

Dynamic context-based routing of speech processing

Features are disclosed for training and using named entity recognition models based on gazetteer information. A named entity recognition model can be trained with a gazetteer output at a layer of the model to provide deterministic data in the probabilistic model. The named entity recognition model can recognize named entities based on the word embedding and the gazetteer output. The named entity recognition model can tune the gazetteer output to include false positive name entities such that the gazetteer output is not deterministic of the output of the model. In some embodiments, the named entity recognition model can be tuned so as to adjust the gazetteer output.

Architecture for gazetteer-augmented named entity recognition

Growing community of inventors

Seattle, WA, United States of America

Wenbo Yan