Systems and methods for automatically annotating multimodal data

Abstract

A system for automatic annotation of multimodal data comprises: an object of interest; a mobile data collection rig comprising an image sensor and a LiDAR sensor; a plurality of mobile beacons located on the object of interest and on the mobile data collection rig; one or more processors and memory. A method for automatic annotation of multimodal data comprises: receiving ultrasonic time of flight data from a plurality of mobile beacons; estimating position data for an object of interest and position data for a mobile data collection rig based on the ultrasonic time of flight data; computing pose information for the object of interest, the mobile data collection rig, an image sensor, and a LiDAR sensor; collecting image data from the image sensor and LiDAR data from the LiDAR sensor; an automatically annotating the image and LiDAR data based on the computed pose information.