Computerized system and method for fine-grained video frame classification and content creation therefrom

Abstract

The disclosed systems and methods provide a novel framework that enables cost-effective, accurate and scalable detection and recognition of key events in sporting or live events. The framework functions by creating a domain-specific video dataset with frame level annotations (i.e., deep domain datasets) and then training a lightweight camera view classifier to detect camera views for a given video. The disclosed framework uses pre-trained pose estimation and panoptic segmentation models along with geometric rules as labeling functions to define scene types and derive frame level classification training data. According to some embodiments, disclosed frameworks may be used to identify key persons or events, select a thumbnail corresponding to a key person or event, generate personalized highlights to enhance user experience and social media promotions for a team, sport or players, and predict and select the best camera view sequence for automatic highlights generation.