Amazon Technologies, Inc.

Prathap Ramachandra

Liu Yang

Jingqian Zhao

Wenbo Yan

Ponnu Jacob

Charlotte Alizerine Dzialo

Ruiqi Luo

Uday Kumar Kollu

Kyung Jae Lee

Lan Ma

Xi Chelsea Chen

Ajay Kumar Soni

Adam Kalman

Yi Qin

Krupal Maddipati

Jinning Wu

Darshan Ashok Fofadiya

Amitpal Singh Bhutani

Nalledath P Vinodkrishnan

Adriano Devillaine

Simone Filice

Meng Xie

Mohammad Alnuaimat

Sean Gunnar Skaar

Lior Maor Maimon

Daksh Gautam

Shu Wang

Deshen Wang

Di Wang

Approaches are generally described for corrupted speech detection in voice-based computer interfaces. First input data including first audio data representing a user utterance may be received. First data representing the first audio data may be generated using a first encoder. First text data representing a transcription of the user utterance may be generated. Second data representing the first text data may be generated using a second encoder different from the first encoder. Third data may be generated by combining the first data and the second data. The third data may be sent to a classifier network trained to predict a relevant corruption state for speech processing inputs. The classifier network may determine that the first input data corresponds to a first corruption state.

Detecting corrupted speech in voice-based computer interfaces

Techniques are described and relate to providing computing services using embeddings of a transformer-based encoder. In an example, a computer system generates, by using a machine learning (ML) transformer, an embedding vector based at least in part on text. The computer system stores the embedding vector and an association between the embedding vector and the text in a data store. Further, the computer system determines that a task is to be performed based at least in part on natural language understanding (NLU) of the text. The computer system receives the embedding vector from the data store based at least in part on the association between the embedding vector and the text. The task is performed based at least in part on the embedding vector after being received from the data store.

Computing services using embeddings of a transformer-based encoder

A multi-tier domain is provided for processing user voice queries and making routing decisions for generating responses, including for user voice queries that include multi-domain trigger words or phrases. When an utterance is recognized as different intents in different domains, a routing system for a domain may consider contextual signals, including those associated with other domains, to determine whether the domain is the proper one to handle the request. This determination can be performed with a statistical model specifically trained to make such determinations using the available contextual data.

Multi-domain intent handling with cross-domain contextual signals

Features are disclosed for training and using a word embedding model configured to receive textual and context data associated with an utterance of a user. A word embedding model can be trained with text data and context data to account for context associated with the text data. The word embedding model can receive an input vector including text data and one or more sets of context data associated with the text data and perform word embedding based on the input vector. In some embodiments, the input vector can include an automatic speech recognition ('ASR') confidence score generated by an ASR model and one or more labels generated by an NLU model. In some embodiments, the input vector can include user information associated with the user.

Architecture for context-augmented word embedding

A speech processing system uses contextual data to determine the specific domains, subdomains, and applications appropriate for taking action in response to spoken commands and other utterances. Some applications may be given priority over others such that some applications are general request applications to which responsibility for processing an intent is to be assigned as long as contextual criteria are satisfied, while other applications are specific request applications to which responsibility for processing an intent is to be assigned only if the applications are specifically requested, if the contextual criteria of priority applications are not satisfied, and/or if certain contextual criteria associated with the specific request applications are satisfied.

Priority and context-based routing of speech processing

A speech processing system uses contextual data to determine the specific domains, subdomains, and applications appropriate for taking action in response to spoken commands and other utterances. The system can use signals and other contextual data associated with an utterance, such as location signals, content catalog data, data regarding historical usage patterns, data regarding content visually presented on a display screen of a computing device when an utterance was made, other data, or some combination thereof.

Dynamic context-based routing of speech processing

A multi-tier architecture is provided for processing user voice queries and making routing decisions for generating responses, including responses to book browsing requests and other content requests. When an utterance is associated with multiple applications in a given domain, the applications may be organized into a subdomain and a tier of routing decisions may be added to the inter-domain and intra-domain routing decision system. The system uses contextual signals to make subdomain routing decisions, including signals regarding content items that are already in a user's content catalog, consumption status of individual content items in the user's catalog, and the like.

Multi-tier speech processing and content operations

Features are disclosed for training and using named entity recognition models based on gazetteer information. A named entity recognition model can be trained with a gazetteer output at a layer of the model to provide deterministic data in the probabilistic model. The named entity recognition model can recognize named entities based on the word embedding and the gazetteer output. The named entity recognition model can tune the gazetteer output to include false positive name entities such that the gazetteer output is not deterministic of the output of the model. In some embodiments, the named entity recognition model can be tuned so as to adjust the gazetteer output.

Architecture for gazetteer-augmented named entity recognition

A new model is introduced into a particular domain that receives a routing of a dialog from a speech processing component. A method associated with the model includes running a set of test utterances through the speech processing component that enables a spoken language dialog with a user to establish a base line score associated with processing for the set of test utterances. The speech processing component determines an intent of the user and routes the spoken language dialog to a network-based domain based on the intent. The method includes establishing an automatic test run of the set of test utterances to obtain a current score and, when a threshold associated with a difference between the current score and the base line score is breached, switching, at the network-based domain, from the false accept detection model to a second model.

Spoken language quality automatic regression detector background

A new model is introduced into a particular domain that receives a routing of a dialog from a speech processing component. The speech processing component is engaged in the dialog with a user and the speech processing component routes the dialog to the particular network-based domain according to a determination by the speech processing component that the user has an intent to perform a task handled by the domain. The model detects, at the domain, whether the user has the proper intent associated with the domain by using the user utterance in its entirety to yield a detection result. When the user does not have the proper intent based on the detection result, the domain drops the user utterance.

Detecting false accepts in a shopping domain for handling a spoken dialog

Growing community of inventors

Seattle, WA, United States of America

Prathap Ramachandra