Neural architecture search using a performance prediction neural network

Abstract

A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described. The method includes obtaining data specifying a current set of candidate architectures for the task neural network; for each candidate architecture in the current set: processing the data specifying the candidate architecture using a performance prediction neural network having multiple performance prediction parameters, the performance prediction neural network being configured to process the data specifying the candidate architecture in accordance with current values of the performance prediction parameters to generate a performance prediction that characterizes how well a neural network having the candidate architecture would perform after being trained on the particular machine learning task; and generating an updated set of candidate architectures by selecting one or more of the candidate architectures in the current set based on the performance predictions for the candidate architectures in the current set.