Prior-driven supervision for weakly-supervised temporal action localization

Abstract

A classifier model is trained for temporal action localization of video clips. A training video clip that includes actions of interest for identification is ingested into the classifier model. Action characteristics within frames of the video clip are identified. The actions correspond to known action classes. An actionness score is determined for each of the frames based upon the action characteristics identified within each of the frames. Class activation sequence (CAS) scores are determined for sequences of the frames based upon a presence or an absence of the action characteristics identified within each of the frames. Base confidence predictions of temporal locations of actions of interest within the video clip are produced by correlating each of the actionness scores with corresponding class activation scores for each of the frames in the sequences of frames. A training class of action snippets corresponding to known ground-truth actions within the video clip is generated based on the base confidence predictions. A classifier is trained with the training class to identify the actions of interest at a video level.