Sensor fusion for object detection

Abstract

Techniques for fusing sensor data generated by different sensor modalities to improve object detections and object predictions determined by low-level systems of a vehicle. The techniques may include determining feature maps based on sensor data generated by different sensor modalities associated with a vehicle. In some examples, the feature maps may include at least a first feature map indicative of a location of an object in an environment of the vehicle and a second feature map indicative of elevation information associated with the object. The techniques may also include inputting the first feature map and the second feature map into a machine-learned model associated with the low-level system of the vehicle. In some examples, an output may be received from the machine-learned model that includes an occupancy grid, and the occupancy grid may exclude representation(s) associated with over-drivable object(s) and/or under-drivable object(s) that may be disposed in the environment.