Generating high-resolution images using self-attention

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating high-resolution images using self-attention based neural networks. One of the systems includes a neural network configured to generate images, the neural network comprising a sequence of one or more first network blocks followed by a sequence of one or more second network blocks, wherein: each first network block is configured to perform operations comprising: applying a self-attention mechanism over at least a subset of first elements of a first block input to generate an updated first block input; and upsampling the updated first block input to generate a first block output; and each second network block is configured to perform operations comprising: processing a second block input using one or more neural network layers to generate an updated second block input; and upsampling the updated second block input to generate a second block output.