Google Inc.

Takaki Makino

Richard Cameron Rose

Otavio Braga

Basilio Garcia Castillo

Hank Liao

Olivier Siohan

Oliver Siohan

Joseph A Nelson

Ryan Mcdonald

Joshua Maynez

Basilio Garcia

Kishan Sachdeva

Basi Garcia

Benyah Shaparenko

A method () includes receiving audio data () corresponding to an utterance () spoken by a user (), receiving video data () representing motion of lips of the user while the user was speaking the utterance, and obtaining multiple candidate transcriptions () for the utterance based on the audio data. For each candidate transcription of the multiple candidate transcriptions, the method also includes generating a synthesized speech representation () of the corresponding candidate transcription and determining an agreement score () indicating a likelihood that the synthesized speech representation matches the motion of the lips of the user while the user speaks the utterance. The method also includes selecting one of the multiple candidate transcriptions for the utterance as a speech recognition output () based on the agreement scores determined for the multiple candidate transcriptions for the utterance.

Rescoring automatic speech recognition hypotheses using audio-visual matching

A method for a privacy-aware transcription includes receiving audio-visual signal including audio data and image data for a speech environment and a privacy request from a participant in the speech environment where the privacy request indicates a privacy condition of the participant. The method further includes segmenting the audio data into a plurality of segments. For each segment, the method includes determining an identity of a speaker of a corresponding segment of the audio data based on the image data and determining whether the identity of the speaker of the corresponding segment includes the participant associated with the privacy condition. When the identity of the speaker of the corresponding segment includes the participant, the method includes applying the privacy condition to the corresponding segment. The method also includes processing the plurality of segments of the audio data to determine a transcript for the audio data.

Privacy-aware meeting room transcription from audio-visual stream

Implementations relate to an application that can bias automatic speech recognition for meetings using data that may be associated with the meeting and/or meeting participants. A transcription of inputs provided during a meeting can additionally and/or alternatively be processed to determine whether the inputs should be incorporated into a meeting document, which can provide a summary for the meeting. In some instances, entries into a meeting document can be designated as action items, and those action items can optionally have conditions for reminding meeting participants about the action items and/or for determining whether an action item has been fulfilled. In this way, various tasks that may typically be manually performed by meeting participants, such as creating a meeting summary, can be automated in a more accurate manner. This can preserve resources that may otherwise be wasted during video conferences, in-person meetings, and/or other gatherings.

Meeting speech biasing and/or document generation based on meeting content and/or related data

Growing community of inventors

Mountain View, CA, United States of America

Takaki Makino