Google Inc.

Basilio Garcia Castillo

Richard Cameron Rose

Otavio Braga

Takaki Makino

Hank Liao

Tara N Sainath

Ruoming Pang

Trevor D Strohman

David Rybach

Oliver Siohan

Olivier Siohan

A method includes receiving a training example that includes audio data representing a spoken utterance and a ground truth transcription. For each word in the spoken utterance, the method also includes inserting a placeholder symbol before the respective word identifying a respective ground truth alignment for a beginning and an end of the respective word, determining a beginning word piece and an ending word piece, and generating a first constrained alignment for the beginning word piece and a second constrained alignment for the ending word piece. The first constrained alignment is aligned with the ground truth alignment for the beginning of the respective word and the second constrained alignment is aligned with the ground truth alignment for the ending of the respective word. The method also includes constraining an attention head of a second pass decoder by applying the first and second constrained alignments.

Emitting word timings with end-to-end models

A method () includes receiving audio data () corresponding to an utterance () spoken by a user (), receiving video data () representing motion of lips of the user while the user was speaking the utterance, and obtaining multiple candidate transcriptions () for the utterance based on the audio data. For each candidate transcription of the multiple candidate transcriptions, the method also includes generating a synthesized speech representation () of the corresponding candidate transcription and determining an agreement score () indicating a likelihood that the synthesized speech representation matches the motion of the lips of the user while the user speaks the utterance. The method also includes selecting one of the multiple candidate transcriptions for the utterance as a speech recognition output () based on the agreement scores determined for the multiple candidate transcriptions for the utterance.

Rescoring automatic speech recognition hypotheses using audio-visual matching

A method for a privacy-aware transcription includes receiving audio-visual signal including audio data and image data for a speech environment and a privacy request from a participant in the speech environment where the privacy request indicates a privacy condition of the participant. The method further includes segmenting the audio data into a plurality of segments. For each segment, the method includes determining an identity of a speaker of a corresponding segment of the audio data based on the image data and determining whether the identity of the speaker of the corresponding segment includes the participant associated with the privacy condition. When the identity of the speaker of the corresponding segment includes the participant, the method includes applying the privacy condition to the corresponding segment. The method also includes processing the plurality of segments of the audio data to determine a transcript for the audio data.

Privacy-aware meeting room transcription from audio-visual stream

Growing community of inventors

Mountain View, CA, United States of America

Basilio Garcia Castillo