Determining geographical map features with multi-sensor input

Abstract

A system trains and applies a machine learning model to label maps of a region. Various data modalities are combined as inputs for multiple data tiles used to characterize a region for a geographical map. Each data modality reflects sensor data captured in different ways. Some data modalities include aerial imagery, point cloud data, and location trace data. The different data modalities are captured independently and then aggregated using machine learning models to determine map labeling information about tiles in the region. Data is ingested by the system and corresponding tiles are identified. A tile is represented by a feature vector of different data types related to the various data modalities, and values from the ingested data are added to the feature vector for the tile. Models can be trained to predict characteristics of a region using these various types of input.