Google Inc.

Daniel M Keysers

Thomas Deselaers

Victor Carbune

Pedro Gonnet Anders

Henry Allan Rowley

Sandro Feuz

Shumin Zhai

Franz Josef Och

Abraham Murray

Neil Matthew Tinmouth Houlsby

Matthew Nirvan Sharifi

Michele Mae Covell

Gheorghe Postelnicu

David Marwood

Ashok Chhabedia Popat

Richard Tucker

Alexey Dosovitskiy

Jakob D Uszkoreit

Alexander Kolesnikov

Lucas Klaus Beyer

Xiaohua Zhai

Thomas Unterthiner

Dmitriy Genzel

Mario Lucic

Basil Mustafa

Joan Puigcerver I Perez

Yin Ching Jessica Yung

Ilya Tolstikhin

Stephan Robert Gammeter

André Susano Pinto

Carlos Riquelme Ruiz

Andreas Peter Steiner

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training neural networks using transfer learning. One of the methods includes training a neural network to perform a first prediction task, including: obtaining trained model parameters for each of a plurality of candidate neural networks, wherein each candidate neural network has been pre-trained to perform a respective second prediction task that is different from the first prediction task; obtaining a plurality of training examples corresponding to the first prediction task; selecting a proper subset of the plurality of candidate neural networks using the plurality of training examples; generating, for each candidate neural network, one or more fine-tuned neural networks, wherein each fine-tuned neural network is generated by updating the model parameters of the candidate neural network using the plurality of training examples; and determining model parameters for the neural network using the respective fine-tuned neural networks.

Training neural networks using transfer learning

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing actionable suggestions are disclosed. In one aspect, a method includes receiving (i) an indication that an event detection module has determined that a shared event of a particular type is presently occurring or has occurred, and (ii) data referencing an attribute associated with the shared event. The method includes selecting, from among multiple output templates that are each associated with a different type of shared event, a particular output template associated with the particular type of shared event detected by the module. The method generates a notification for output using at least (i) the selected particular output template, and (ii) the data referencing the attribute associated with the shared event. The method then provides, for output to a user device, the notification that is generated.

Actionable suggestions for activities

Systems and methods enable a computing system to recognize a sequence of repeated actions and offer to automatically repeat any such recognized actions. An example method includes determining a current sequence of user actions is similar to previous sequence of user actions, determining whether the previous sequence is reproducible and, when reproducible, initiating display of a prompt that requests approval for completing the current sequence based on the previous sequence and, responsive to receiving an indication of approval, completing the previous sequence. Another example method included determining that a first current sequence of user interactions is complete and is not similar to any saved sequence of user interactions, saving the first current sequence as a previous sequence, identifying a second sequence as satisfying a similarity threshold with the previous sequence, and initiating display of a prompt that requests approval for saving the previous sequence as a shortcut.

Smart assist for repeated actions

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using mixer neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more mixer neural network layers.

Multi-layer perceptron-based computer vision neural networks

A method is provided that can include determining, based on a plurality of media streams of a plurality of client devices participating in a video conference, a subject being discussed during the video conference at a point in time, and providing one or more inputs to a machine learning model, the one or more inputs comprising a plurality of transcriptions each corresponding to one of the plurality of media streams, and an indication of a request to identify one or more media streams that are relevant the subject. The method can further include determining, based on one or more outputs of the machine learning model, a media stream that is most relevant to the subject, and providing the determined media stream for presentation on one or more of the plurality of client devices while the subject is being discussed during the video conference.

Video conference content auto-retrieval and focus based on learned relevance

Systems and methods for video conference content auto-retrieval and focus based on learned relevance is provided. A method may include determining collaborative documents each associated with at least one user participating in a video conference, and providing one or more first inputs to a machine learning model, the one or more first inputs identifying the plurality of collaborative documents, and comprising an indication of a request to identify a collaborative document that is most relevant to a first subject discussed during the video conference at a first point in time. The method may further include determining, based on one or more first outputs of the machine learning model, the collaborative document that is the most relevant to the first subject discussed during the video conference at the first point in time, and providing the determined collaborative document for presentation on one or more client devices while the first subject is being discussed during the video conference.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that use generative adversarial models to increase the quality of sensor data generated by a first environmental sensor to resemble the quality of sensor data generated by another sensor having a higher quality than the first environmental sensor. A set of first and second training data generated by a first environmental sensor having a first quality and a second sensor having a target quality, respectively, is received. A generative adversarial mode is trained, using the set of first training data and the set of second training data, to modify sensor data from the first environmental sensor by reducing a difference in quality between the sensor data generated by the first environmental sensor and sensor data generated by the target environmental sensor.

Modifying sensor data using generative adversarial models

Systems and methods for video conference content auto-retrieval and focus based on learned relevance is provided. In accordance with the systems and methods, audio streams and video streams from client devices participating in a video conference are received. Based on the audio streams, a subject being discussed during the video conference at a point in time is determined. A video stream that is most relevant to the subject being discussed during the video conference at the point in time is determined from the video streams. The determined video stream is provided to the client devices for presentation on the client devices while the subject is being discussed during the video conference.

Methods, apparatus, and computer readable media related to receiving textual input of a user during a dialog between the user and an automated assistant (and optionally one or more additional users), and generating responsive reply content based on the textual input and based on user state information. The reply content is provided for inclusion in the dialog. In some implementations, the reply content is provided as a reply, by the automated assistant, to the user's textual input and may optionally be automatically incorporated in the dialog between the user and the automated assistant. In some implementations, the reply content is suggested by the automated assistant for inclusion in the dialog and is only included in the dialog in response to further user interface input.

Using textual input and user state information to generate reply content to present in response to the textual input

Techniques are described herein for enabling an automated assistant to adjust its behavior depending on a detected age range and/or 'vocabulary level' of a user who is engaging with the automated assistant. In various implementations, data indicative of a user's utterance may be used to estimate one or more of the user's age range and/or vocabulary level. The estimated age range/vocabulary level may be used to influence various aspects of a data processing pipeline employed by an automated assistant. In various implementations, aspects of the data processing pipeline that may be influenced by the user's age range/vocabulary level may include one or more of automated assistant invocation, speech-to-text ('STT') processing, intent matching, intent resolution (or fulfillment), natural language generation, and/or text-to-speech (&#x201c;TTS&#x201d;) processing. In some implementations, one or more tolerance thresholds associated with one or more of these aspects, such as grammatical tolerances, vocabularic tolerances, etc., may be adjusted.

Automated assistants that accommodate multiple age groups and/or vocabulary levels

Techniques are described herein for enabling an automated assistant to adjust its behavior depending on a detected vocabulary level or other vocal characteristics of an input utterance provided to an automated assistant. The estimated vocabulary level or other vocal characteristics may be used to influence various aspects of a data processing pipeline employed by the automated assistant. In some implementations, one or more tolerance thresholds associated with, for example, grammatical tolerances or vocabulary tolerances, may be adjusted based on the estimated vocabulary level or vocal characteristics of the input utterance.

Growing community of inventors

Zurich, Switzerland

Daniel M Keysers