Amazon Technologies, Inc.

Frederick Victor Weber

Baiyang Liu

Hugh Evan Secker-Walker

Jeffrey Penrod Adams

Stan Weidner Salvador

James G Droppo

Ariya Rastrow

Ryan Paul Thomas

Jeffrey C O'Neill

Jeffrey Paul Lilly

Venkata Kishore Nandury

Andrew Oberlin

Pankaj Sitpure

Gurpreet Singh Chadha

Milind Mukesh Rao

Santosh Kumar Patro

Buddha Puneeth Nandanoor

Anand Mohan

Gopinath Chennupati

Prahalad Venkataramanan

A first neural network (NN) model may generate labels for training a second NN model. The second NN model may represent instances of a NN model operating on multiple different devices (e.g., decentralized user and/or edge devices). The system may include using a 'teacher' model to process data received by one or more of the devices to generate a labeled dataset. The system may use the labeled dataset and a “student” model to calculate gradient data for updating the student model. The student model may be the same or similar to NN model instances operating on the devices. The system may validate the updated student model to determine, for example, whether it exhibits improved performance when processing the newly received data and/or historical data. The system may distribute the validated update to the devices.

Continuous learning for machine learning models

Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition ('ASR') and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models (“GMMs”) that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

Automatic speaker identification using speech recognition features

Features are disclosed for automatically identifying a speaker. Artifacts of automatic speech recognition ('ASR') and/or other automatically determined information may be processed against individual user profiles or models. Scores may be determined reflecting the likelihood that individual users made an utterance. The scores can be based on, e.g., individual components of Gaussian mixture models ('GMMs') that score best for frames of audio data of an utterance. A user associated with the highest likelihood score for a particular utterance can be identified as the speaker of the utterance. Information regarding the identified user can be provided to components of a spoken language processing system, separate applications, etc.

A home interface comprises one or more audio interfaces within the premises of users. The audio interfaces are connected to an assistant service that detects conditions within rooms of user premises and that generates simulated audio themes for the rooms. The audio themes may be changed depending on detected conditions, and subthemes of a given audio theme may be implemented in different rooms.

Audio themes

Features are disclosed for generating acoustic models from an existing corpus of data. Methods for generating the acoustic models can include receiving at least one characteristic of a desired acoustic model, selecting training utterances corresponding to the characteristic from a corpus comprising audio data and corresponding transcription data, and generating an acoustic model based on the selected training utterances.

Acoustic model training

An unsupervised acoustic modeling service for speech recognition is disclosed. A computing device may be present in a listening zone, such as a household or office, and may receive an audio signal that may include speech. In some instances, the speech is not directed to the computing device. Speech recognition results may be generated from the audio signal, using an acoustic model, and the results used to update the acoustic model. For example, the acoustic model may be updated to reflect a particular speaker's pronunciation of certain sound units, such as phonemes. By using speech that is not necessarily directed to the computing device, more data may be available for updating an acoustic model.

Unsupervised acoustic model training

A language model for speech recognition may be discriminatively pruned. In some embodiments, the language model is discriminatively pruned by computing a discriminative objective function value for one or more n-grams in the language model, and selecting one or more n-grams to prune based at least in part on a threshold value. In some embodiments, the language model is discriminatively pruned to a sufficiently small number of n-grams such that transcription of audio inputs may occur in real time, or such that the pruned language model may be stored on a device with relatively limited electronic storage capacity.

Discriminative language model pruning

Natural language controlled devices may be configured to activate command recognition in response to one or more wake words. Techniques are provided to receive a candidate word for evaluation as a wake word that activates a natural language control functionality of a computing device. The candidate word may include one or more words or sounds. Values for multiple wake word metrics are then determined. The candidate word is evaluated based on the various wake word metrics.

Wake word evaluation

Features are disclosed for discriminative training of speech recognition language models. A confusion matrix can be generated from acoustic model training data for use in discriminative training. The confusion matrix can include probabilities for the substitution, insertion, and/or deletion of some or all subword units of a language. Probabilities can be calculated based on the presence or absence of subword units in a processed acoustic model training data audio recording when compared to a correct transcription of the recording. The probabilities can be used to generate erroneous transcriptions in language model training corpora, and the language model can be trained to distinguish the erroneous transcriptions from the correct transcriptions.

Discriminative language model training using a confusion matrix

Neural networks may be used in certain automatic speech recognition systems. To improve performance of these neural networks, they may be updated/retrained during run time by training the neural network based on the output of a speech recognition system or based on the output of the neural networks themselves. The outputs may include weighted outputs, lattices, weighted N-best lists, or the like. The neural networks may be acoustic model neural networks or language model neural networks. The neural networks may be retrained after each pass through the network, after each utterance, or in varying time scales.

Growing community of inventors

New York, NY, United States of America

Frederick Victor Weber