Google Inc.

Alexander Jay Cuthbert

Joshua James Estelle

Sunny Goyal

Masakazu Seno

Barak Turovsky

Matthew Morton Gaba

Peng Xu

Macduff Richard Hughes

Chao Tian

John Sturdy DeNero

Arne Mauser

Minqi Sebastian Jiang

Julie Cattiau

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for presenting additional information for text depicted by an image. In one aspect, a method includes receiving an image. Text depicted in the image is identified. The identified text can be in one or more text blocks. A prominence presentation context is selected for the image based on the relative prominence of the one or more text blocks. Each prominence presentation context corresponds to a relative prominence of each text block in which text is presented within images. Each prominence presentation context has a corresponding user interface for presenting additional information related to the identified text depicted in the image. A user interface is identified that corresponds to the selected prominence presentation context. Additional information is presented for at least a portion of the text depicted in the image using the identified user interface.

Presenting translations of text depicted in images

Computer-implemented techniques can include receiving a selected word in a source language, obtaining one or more parts of speech for the selected word, and for each of the one or more parts-of-speech, obtaining candidate translations of the selected word to a different target language, each candidate translation corresponding to a particular semantic meaning of the selected word. The techniques can include for each semantic meaning of the selected word: obtaining an image corresponding to the semantic meaning of the selected word, and compiling translation information including (i) the semantic meaning, (ii) a corresponding part-of-speech, (iii) the image, and (iv) at least one corresponding candidate translation. The techniques can also include outputting the translation information.

Techniques for providing visual translation cards including contextually relevant definitions and examples

A language translation application on a user device includes a user interface that provides relevant textual and graphical feedback mechanisms associated with various states of voice input and translated speech.

User interface for realtime language translation

A computer-implemented technique can include receiving a selection by a user of a single word in a document in a source language, the document being displayed in a viewing application executing at the computing device, obtaining contextual information from the document that is indicative of a context of the selected word, providing the selected word and its contextual information from the viewing application to a different translation application, obtaining potential translated words using the translation application, the selected word, and its contextual information, each potential translated word being a potential translation of the selected word to a different target language that is preferred by the user, and displaying the potential translated words.

Coordinated user word selection for translation and obtaining of contextual information for the selected word

Computer-implemented techniques can include obtaining languages for automatic speech recognition, the languages being associated with a plurality of users and receiving an input indicative of a request to provide speech input. In response to receiving the input, a first indication that automatic speech recognition (ASR) is being performed in each of the plurality of languages can be displayed. A speech input from one of the plurality of users can be received and ASR can identify one of the languages associated with the speech input to obtain a detected language. In response to obtaining the detected language, a second indication that ASR is being performed in the detected language can be displayed. The speech input can be transcribed and translated to another one of the plurality of languages to obtain a translated text that can be displayed.

Automatic speech recognition techniques for multiple languages

A computer-implemented technique includes techniques are presented for user image capture feedback for improved machine language translation. When machine language translation of OCR text obtained from an initial image has a low degree of likelihood of being an appropriate translation, these techniques provide for user image capture feedback to obtain additional images to obtain a modified OCR text, which can result in improved machine language translation results. Instead of user image capture feedback, the techniques may obtain the modified OCR text by selecting another possible OCR text from the initial OCR operation. In addition to additional image capturing, light source intensity and/or a quantity/number of light source flashes can be adjusted. After obtaining the modified OCR text, another machine language translation can be obtained and, if it has a high enough degree of likelihood, it can then be output to a user.

Techniques for providing user image capture feedback for improved machine language translation

Computer-implemented techniques can include capturing, by a microphone associated with a computing device having one or more processors, a speech input from a user, the speech input comprising a single word in a source language, and in response to receiving the speech input from the user, performing a plurality of actions. The plurality of actions can include identifying, by the computing device, the source language of the single word and a target language that is associated with the user, obtaining, by the computing device, one or more translated words that are each a potential translation of the single word to the target language, obtaining, by the computing device, lexicon data for the one or more translated words, the lexicon data relating to at least one semantic meaning of the one or more translated words, and displaying, by the computing device, the lexicon data.

Techniques for providing lexicon data for translation of a single word speech input

Display screen with graphical user interface

A computer-implemented technique can include receiving a media feed from a speaker computing device representing speech of a speaker user captured by the speaker computing device. The technique can include receiving a plurality of translation requests, each translation request being received from a listener computing device associated with a listener user and corresponding to a request to obtain a translated version of the media feed into a preferred language of the listener user. The technique can include determining the preferred language for each listener user. The technique can include obtaining a machine translated media feed for each of the translation requests, the machine translated media feed corresponding to a translation of the media feed from the source language to the preferred language of the listener user associated with the translation request. The technique can also include outputting the machine translated media feeds to the listener computing devices.

Techniques for real-time translation of a media feed from a speaker computing device and distribution to multiple listener computing devices in multiple different languages

A computer-implemented technique can include receiving a machine translation input specifying (i) a source text, (ii) a source language of the source text, and (iii) a target language for the source text, and obtaining a machine translation of the source text from the source language to the target language to obtain a translated source text. The technique can include determining whether to swap the source and target languages based on (i) the source text and (ii) at least one language model, and in response to determining to swap the source and target languages: swapping the source and target languages to obtain modified source and target languages, utilizing the translated source text as a modified source text, obtaining a machine translation of the modified source text from the modified source language to the modified target language to obtain a translated modified source text, and outputting the translated modified source text.

Growing community of inventors

Oakland, CA, United States of America

Alexander Jay Cuthbert