Semi-supervised training of neural networks

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes obtaining a batch of labeled training items and a batch of unlabeled training items; processing the labeled training items and the unlabeled training items using the neural network and in accordance with current values of the network parameters to generate respective embeddings; determining a plurality of similarity values, each similarity value measuring a similarity between the embedding for a respective labeled training item and the embedding for a respective unlabeled training item; determining a respective roundtrip path probability for each of a plurality of roundtrip paths; and performing an iteration of a neural network training procedure to determine a first value update to the current values of the network parameters that decreases roundtrip path probabilities for incorrect roundtrip paths.