Method for dynamic 3d crowd reconstruction from a large-scene video

Abstract

This invention focuses on the 3D reconstruction of dynamic crowds in large-scene videos and introduces the DyCrowd framework, which reconstructs 3D position, pose, and shape of hundreds of people from a large-scene video. Our approach addresses frequent occlusions and modeling challenges in high-density crowds through a top-down strategy. This includes pre-reconstruction, matching individual movement sequences, and multi-stage iterative optimization. During the optimization process, we introduce a group optimization method with an asynchronous motion consistency loss. This method clusters individuals with similar trajectories, using high-quality and unoccluded movements within the group to guide the recovery of occluded individuals, thereby mitigating long-term occlusion issues. Furthermore, to address the lack of ground-truth human reconstruction labels in current large-scene datasets, we introduce a virtual benchmark dataset called VirtualCrowd for dynamic crowd reconstruction in large-scene videos.