Deep association for sensor fusion

Abstract

This document describes systems and techniques related to deep association for sensor fusion. For example, a model trained using deep machine learning techniques, may be used to generate an association score matrix that includes probabilities that tracks from different types of sensors are related to the same objects. This model may be trained using a convolutional recurrent neural network and include constraints not included in other training techniques. Focal loss can be used during training to compensate for imbalanced data samples and address difficult cases, and data expansion techniques can be used to increase the multi-sensor data space. Simple thresholding techniques can be applied to the association score matrix to generate an assignment matrix that indicates whether tracks from one sensor and tracks from another sensor match. In this manner, the track association process may be more accurate than current sensor fusion techniques, and vehicle safety may be increased.