Self-learning object detectors for unlabeled videos using multi-task learning

Abstract

A system and method enable generating a specific object detector for a category of interest. The method includes identifying seed objects in frames of a video sequence with a pre-trained generic detector for the category. An appearance model is iteratively learned for each of the seed objects using other frames in which the seed object is identified. The appearance models are learned jointly to optimize a loss function which accounts for the loss of incorrectly labeling sub-images and a regularization term which measures a distance between the appearance models. The loss of incorrectly labeling sub-images is determined using a motion model which predicts the location of the seed object in the subsequent frames so that sub-images outside the location that the current appearance model contribute to the loss. The specific object detector is then generated by aggregating the optimized appearance models.