Amazon Technologies, Inc.

Denis Sergeyevich Filimonov

Ariya Rastrow

Gautam Tiwari

Shaun Nidhiri Joseph

Ankur Gandhe

Rohit Prasad

Spyridon Matsoukas

Markus Dreyer

Björn Hoffmeister

Nikko Ström

Ravi Teja Gadde

Yuan Shangguan

Julian Chan

Techniques for performing automatic speech recognition (ASR) processing are described. The ASR processing may involve use of a segmenter and a decoder. The segmenter may be configured to identify audio segments, from audio data containing speech, based on word boundaries. The decoder may be configured to generate multiple word hypotheses for individual audio segments. A fixed-size representation of an audio segment may be generated. The described ASR processing techniques may be computationally less expensive than at least some other systems. Also, the described ASR processing techniques may maintain a larger number of word predictions per audio segment as compared to at least some other systems.

Automatic speech recognition (ASR) techniques

This disclosure proposes systems and methods for leveraging entity-related language models in speech processing. A system can receive audio data corresponding to an utterance and perform automatic speech recognition (ASR) on a first portion of the audio data using a general language model. Based on the results, the system may identify a specific language model for processing a second portion of the audio data. The specific language model may include entities belonging to a common subject or class. The specific language model may, in some cases, provide better results than the general language model. While the general language model may describe a whole sentence, the specific language model may describe only a portion of a sentence. Thus, a top-level model may 'activate' the specific language model when it may provide useful results. The resulting data may include results from both the general language model and the specific language model.

Entity language models for speech processing

Compact finite state transducers (FSTs) for automatic speech recognition (ASR). An HCLG FST and/or G FST may be compacted at training time to reduce the size of the FST to be used at runtime. The compact FSTs may be significantly smaller (e.g., 50% smaller) in terms of memory size, thus reducing the use of computing resources at runtime to operate the FSTs. The individual arcs and states of each FST may be compacted by binning individual weights, thus reducing the number of bits needed for each weight. Further, certain fields such as a next state ID may be left out of a compact FST if an estimation technique can be used to reproduce the next state at runtime. During runtime portions of the FSTs may be decompressed for processing by an ASR engine.

Compressed finite state transducers for automatic speech recognition

A reduced latency system for automatic speech recognition (ASR). The system can use certain feature values describing the state of ASR processing to estimate how far a lowest scoring node for an audio frame is from a potential node likely be part of the Viterbi path. The system can then adjust its beam width in a manner likely to encompass the node likely to be on the Viterbi path, thus pruning unnecessary nodes and reducing latency. The feature values and estimated distances may be based on a set of training data, where the system identifies specific nodes on the Viterbi path and determines what feature values correspond to what desired beam widths. Trained models or other data may be created at training and used at runtime to dynamically adjust the beam width, as well as other settings such as threshold number of active nodes.

Adaptive beam pruning for automatic speech recognition

Features are disclosed for performing speech recognition on utterances using dynamic weights with speech recognition models. An automatic speech recognition system may use a general speech recognition model, such a large finite state transducer-based language model, to generate speech recognition results for various utterances. The general speech recognition model may include sub-models or other portions that are customized for particular tasks, such as speech recognition on utterances regarding particular topics. Individual weights within the general speech recognition model can be dynamically replaced based on the context in which an utterance is made or received, thereby providing a further degree of customization without requiring additional speech recognition models to generated, maintained, or loaded.

Dynamic arc weights in speech recognition models

A language model for automatic speech processing, such as a finite state transducer (FST) may be configured to incorporate information about how a particular word sequence (N-gram) may be used in a similar manner from another N-gram. A score of a component of the FST (such as an arc or state) relating to the first N-gram may be based on information of the second N-gram. Further, the FST may be configured to have an arc between a state of the first N-gram and a state of the second N-gram to allow for cross N-gram back off, rather than backoff from a larger N-gram to a smaller N-gram during traversal of the FST during speech processing.

Automatic speech recognition incorporating word usage information

An automatic speech recognition ('ASR') system produces, for particular users, customized speech recognition results by using data regarding prior interactions of the users with the system. A portion of the ASR system (e.g., a neural-network-based language model) can be trained to produce an encoded representation of a user's interactions with the system based on, e.g., transcriptions of prior utterances made by the user. This user-specific encoded representation of interaction history is then used by the language model to customize ASR processing for the user.

Growing community of inventors

Seattle, WA, United States of America

Denis Sergeyevich Filimonov