Google Inc.

Basil Mustafa

Neil Matthew Tinmouth Houlsby

Joan Puigcerver I Perez

Vivek Natarajan

Sylvain Gelly

Lucas Klaus Beyer

Alexander Kolesnikov

Xiaohua Zhai

Michael Tobias Tschannen

Yin Ching Jessica Yung

Alan Prasana Karthikesalingam

Jan Freyberg

Wen Yau Aaron Loh

Patricia Leigh MacWilliams

André Susano Pinto

Carlos Riquelme Ruiz

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training neural networks using transfer learning. One of the methods includes training a neural network to perform a first prediction task, including: obtaining trained model parameters for each of a plurality of candidate neural networks, wherein each candidate neural network has been pre-trained to perform a respective second prediction task that is different from the first prediction task; obtaining a plurality of training examples corresponding to the first prediction task; selecting a proper subset of the plurality of candidate neural networks using the plurality of training examples; generating, for each candidate neural network, one or more fine-tuned neural networks, wherein each fine-tuned neural network is generated by updating the model parameters of the candidate neural network using the plurality of training examples; and determining model parameters for the neural network using the respective fine-tuned neural networks.

Training neural networks using transfer learning

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform a downstream computer vision task. One of the methods includes pre-training an initial neural network that shares layers with the neural network to perform an initial computer vision task and then training the neural network on the downstream computer vision task.

Transfer learning between different computer vision tasks

Growing community of inventors

Zurich, Switzerland

Basil Mustafa