Method and system for generating a training dataset for keypoint detection, and method and system for predicting 3d locations of virtual markers on a marker-less subject

Abstract

According to embodiments of the present invention, a method and system for generating a training dataset for keypoint detection are provided. The system includes an optical marker-based motion capture system to capture markers as 3D trajectories; and video cameras to simultaneously capture sequences of 2D images. Each marker is placed on a bone landmark or keypoint of a subject. The method, performed by a computer in the system, includes projecting each trajectory to each image to determine a 2D location for each marker; interpolating a 3D position therefrom; generating a bounding box around the subject; and generating the training dataset including at least one image, and the determined 2D location of each marker and the bounding box therein. According to further embodiments, a method and system for predicting 3D locations of virtual markers on a marker-less subject using a neural network trained by the generated training dataset are also provided.