Spatio-temporal awareness engine for priority tree based region selection across multiple input cameras and multimodal sensor empowered awareness engine for target recovery and object path prediction

Abstract

A spatio-temporal awareness engine combines a low-resolution tracking process and high resolution tracking process to employ an array of imaging sensors to track an object within the visual field. The system utilizes a low-resolution conversion through noise filtering and feature consolidation to load-balance the more computationally-intensive aspects of object tracking, allowing for a more robust system, while utilizing less computer resources. A process for target recovery and object path prediction in a robotic drone may include tracking targets using a combination of visual and acoustic multimodal sensors, operating a camera as a main tracking sensor of the multimodal sensors and feeding output of the camera to a spatiotemoral engine, complementing the main tracking sensor with non-visual, fast secondary sensors to assign rough directionality to a target tracking signal, and applying the rough directionality to prioritize visual scanning by the main tracking sensor.