Autonomous vehicle perception multimodal sensor data management

Abstract

The automated driving perception systems described herein provide technical solutions for technical problems facing navigation sensors for autonomous vehicle navigation. These systems may be used to combine inputs from multiple navigation sensors to provide a multimodal perception system. These multimodal perception systems may augment raw data within a development framework to improve performance of object detection, classification, tracking, and sensor fusion under varying external conditions, such as adverse weather and light, as well as possible sensor errors or malfunctions like miss-calibration, noise, and dirty or faulty sensors. This augmentation may include injection of noise, occlusions, and misalignments from raw sensor data, and may include ground-truth labeling to match the augmented data. This augmentation provides improved robustness of the trained perception algorithms against calibration, noise, occlusion, and faults that may exist in real-world scenarios.