System and method for automatic detection of visual events in transportation environments

Abstract

This invention provides a system and method that uses a hybrid model for transportation-based (e.g. maritime) visual event detection of events. In operation, video data is reduced by detecting change and exclusively transmitting images to the deep learning model when changes are detected, or alternatively, based upon a timer that samples at selected intervals. Relatively straightforward deep learning models are used, which operate on sparse individual frames, instead of employing complex deep learning models that operate on multiple frames/videos. This approach reduces the need for specialized models. Independent, rule-based classifiers are used, based on the output of the deep learning model into visual events that, in turn, allows highly specialized events to be constructed. For example, multiple detections can be combined into higher-level single events, and thus, the existence maintenance procedures, cargo activities, and/or inspection rounds can be derived from combining multiple events or multiple detections.