Semantic parsing of objects in video

Abstract

The invention provides an improved method to detect semantic attributes of human body in computer vision. In detecting semantic attributes of human body in computer vision, the invention maintains a list of semantic attributes, each of which corresponds to a human body part. A computer module then analyzes segments of a frame of a digital video to detect each semantic attribute by finding a most likely attribute for each segment. A threshold is applied to select candidate segments of the frame for further analysis. The candidate segments of the frame then go through geometric and resolution context analysis by applying the physical structure principles of a human body and by analyzing increasingly higher resolution versions of the image to verify the existence and accuracy of parts and attributes. A computer module computes a resolution context score for a lower resolution version of the image based on a weighted average score computed for a higher resolution version of the image by evaluating appearance features, geometric features, and resolution context features when available on the higher resolution version of the image. Finally, an optimal configuration step is performed via dynamic programming to select an optimal output with both semantic attributes and spatial positions of human body parts on the frame.