Google Inc.

Golan Pundak

Dimitri Kanevsky

Tara N Sainath

Rohit Prakash Prabhavalkar

Antoine Jean Bruguier

Ke Hu

Tomer Amiaz

Asaf Aharoni

Assaf Hurwitz Michaely

Dan Rasin

Rafael Goldfarb

Lior Alon

Charles Caleb Peyser

Shmuel Leibtag

Or Guz

Various arrangements for enhancing audio are detailed herein. An audio stream and a second audio stream can be received. From these audio streams, a first audio source and a second audio source are extracted. A conversation between the first audio source and a third audio source that occurs within the audio streams is identified. An updated audio stream is generated that enhances the first audio source and diminishes the second audio source extracted from the audio stream and the second audio stream.

Enhancing audio using multiple recording devices

A method includes receiving audio data encoding an utterance and obtaining a set of bias phrases corresponding to a context of the utterance. Each bias phrase includes one or more words. The method also includes processing, using a speech recognition model, acoustic features derived from the audio to generate an output from the speech recognition model. The speech recognition model includes a first encoder configured to receive the acoustic features, a bias encoder configured to receive data indicating the obtained set of bias phrases, a bias encoder, and a decoder configured to determine likelihoods of sequences of speech elements based on output of the first attention module and output of the bias attention module. The method also includes determining a transcript for the utterance based on the likelihoods of sequences of speech elements.

Contextual biasing for speech recognition

In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for identifying that a first audio stream includes first, second, and third sources of audio. A computing system identifies that a second audio stream includes the first, second, and third sources of audio. The computing system determines that the first and second sources of audio are part of a first conversation. The computing system generates a third audio stream that combines the first source of audio from the first audio stream, the first source of audio from the second audio stream, the second source of audio from the first audio stream, and the second source of audio from the second audio stream, and diminishes the third source of audio from the first audio stream, and the third source of audio from the second audio stream.

A method includes receiving audio data encoding an utterance spoken by a native speaker of a first language, and receiving a biasing term list including one or more terms in a second language different than the first language. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data to generate speech recognition scores for both wordpieces and corresponding phoneme sequences in the first language. The method also includes rescoring the speech recognition scores for the phoneme sequences based on the one or more terms in the biasing term list, and executing, using the speech recognition scores for the wordpieces and the rescored speech recognition scores for the phoneme sequences, a decoding graph to generate a transcription for the utterance.

Phoneme-based contextualization for cross-lingual speech recognition in end-to-end models

Implementations are directed to causing a voice bot to utilize a plurality of ML layers in resolving unique personal identifier(s) for a human while the voice bot is engaged in a corresponding conversation with the human. The unique personal identifier(s) can include a unique sequence of alphanumeric characters that is personal to the human. In some implementations, ASR speech hypothes(es) corresponding to spoken utterance(s) that include the unique personal identifier(s) can be processed to generate candidate unique personal identifier(s), given alphanumeric character(s) of the candidate unique personal identifier(s) can be selected, and the voice bot can prompt the human with clarification request(s) to clarify the given alphanumeric character(s) until it is predicted to correspond to the an actual unique personal identifier(s) for the human(s). The unique personal identifier(s) can then be utilized in performance of further action(s) by the voice bot and/or other systems.

Resolving unique personal identifiers during corresponding conversations between a voice bot and a human

A method for training a speech recognition model with a minimum word error rate loss function includes receiving a training example comprising a proper noun and generating a plurality of hypotheses corresponding to the training example. Each hypothesis of the plurality of hypotheses represents the proper noun and includes a corresponding probability that indicates a likelihood that the hypothesis represents the proper noun. The method also includes determining that the corresponding probability associated with one of the plurality of hypotheses satisfies a penalty criteria. The penalty criteria indicating that the corresponding probability satisfies a probability threshold, and the associated hypothesis incorrectly represents the proper noun. The method also includes applying a penalty to the minimum word error rate loss function.

Proper noun recognition in end-to-end speech recognition

A method of biasing speech recognition includes receiving audio data encoding an utterance and obtaining a set of one or more biasing phrases corresponding to a context of the utterance. Each biasing phrase in the set of one or more biasing phrases includes one or more words. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data and grapheme and phoneme data derived from the set of one or more biasing phrases to generate an output of the speech recognition model. The method also includes determining a transcription for the utterance based on the output of the speech recognition model.

A method includes receiving audio data encoding an utterance and obtaining a set of bias phrases corresponding to a context of the utterance. Each bias phrase includes one or more words. The method also includes processing, using a speech recognition model, acoustic features derived from the audio to generate an output from the speech recognition model. The speech recognition model includes a first encoder configured to receive the acoustic features, a first attention module, a bias encoder configured to receive data indicating the obtained set of bias phrases, a bias encoder, and a decoder configured to determine likelihoods of sequences of speech elements based on output of the first attention module and output of the bias attention module. The method also includes determining a transcript for the utterance based on the likelihoods of sequences of speech elements.

Contextual biasing for speech recognition using grapheme and phoneme data

Methods, systems, and apparatus are described for inducing a user of a speech recognition system to adjust their own behavior. For example, in one implementation, a speech recognition system that allows children to control electronic devices can improve the child's speech development, by encouraging the child to speak more clearly. To do so, the speech recognition system can generate a phonetic representation of a term spoken by the child, and can determine whether the phonetic representation matches a particular canonical pronunciation of the particular term that is deemed age-appropriate for the child. Upon determining that the particular canonical pronunciation that matches the phonetic representation of the term spoken by the child is not age-appropriate, the speech recognition system can select and implement a variety of remediation strategies for inducing the child to repeat the term using a pronunciation that is considered age-appropriate.

Growing community of inventors

New York, NY, United States of America

Golan Pundak