Neural radiance fields for orthographic imagery

Abstract

Methods and systems for generating orthographic imagery are provided. An example method involves accessing a plurality of source images depicting a scene from multiple points of view, encoding at least some of the source images into corresponding series of multiscale feature maps, receiving a definition of an orthographic projection of the scene, and generating an orthographic image of the scene corresponding to the orthographic projection. The generating involves applying global attention across a set of higher-level features, applying a convolutional and upsampling layer, generating a depth map for the scene, and back-projecting each point on the depth map corresponding to a pixel that should be rendered back through the orthographic projection to determine a set of lower-level features to be used to decode the pixel, and applying local attention across the set of lower-level features to produce a set of decoded features from which pixel information can be determined.