Google Inc.

Afroz Mohiuddin

Nitin Kumar Gupta

Harshit Kharbanda

Jessica Lee

Preyas Popat

Dounia Berrada

Samer Hassan Hassan

Ali Essam Ali Elqursh

Mikhail Khalman

Lukasz Mieczyslaw Kaiser

Christopher Kelley

Belinda Zeng

Yi Liu

Ni Lao

Fabian Roth

The present disclosure provides computer-implemented methods, systems, and devices for responding to requests associated with an image. A computing system obtains, wherein the image depicts a first set of textual content. The computing system determines one or more characteristics of the first set of textual content. The computing system determines a response type from a plurality of response types based on the one or more characteristics. The computing system generates a model input, wherein the model input comprises data descriptive of the first set of textual content and a prompt associated with the response type. The computing system provides providing the model input as an input to a machine-learned language model. The computing system receives a second set of text as an output of the machine-learned language model as a result of the machine-learned language model processing the model input. The computing system provides the second set of text for display to a user, wherein the second set of textual content is associated with the response type.

Systems and methods for analyzing text extracted from images and performing appropriate transformations on the extracted text

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for identifying answers to questions using neural networks. One of the methods includes receiving an input text passage and an input question string; processing the input text passage using an encoder neural network to generate a respective encoded representation for each passage token in the input text passage; at each time step: processing a decoder input using a decoder neural network to update the internal state of the decoder neural network; and processing the respective encoded representations and a preceding output of the decoder neural network using a matching vector neural network to generate a matching vector for the time step; and generating an answer score that indicates how well the input text passage answers a question posed by the input question string.

Answer to question neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating answers to answer-seeking queries. One of the methods includes receiving a query having multiple terms. The query is classified as an answer-seeking query of a particular question type, and one or more answer types associated with the particular question type are obtained. Search results satisfying the query are obtained, and a respective score is computed for each of one or more passages of text occurring in each document identified by the search results, wherein the score for each passage of text is based on how many of the one or more answer types match the passage of text. A presentation that includes information from one or more of the passages of text selected based on the respective score is provided in response to the query.

Generating elements of answer-seeking queries and elements of answers

Growing community of inventors

San Jose, CA, United States of America

Afroz Mohiuddin