System and method for autoencoder based multi-sensor fusion

Abstract

Multi-sensor fusion is a technology which effectively utilizes the data from multiple sensors so as to portray a unified picture with improved information and offers significant advantages over existing single sensor-based techniques. This disclosure relates to a method and system for a multi-label classification using a two-stage autoencoder. Herein, the system employs autoencoder based architectures, where either raw sensor data or hand-crafted features extracted from each sensor are used to learn sensor-specific autoencoders. The corresponding latent representations from a plurality of sensors are combined to learn a fusing autoencoder. The latent representation of the fusing autoencoder is used to learn a label consistent classifier for multi-class classification. Further, a joint optimization technique is presented for learning the autoencoders and classifier weights together. Herein, the joint optimization allows discriminative features to be learnt from the plurality of sensors and hence it displays superior performance than the state-of-the-art methods with reduced complexity.