Activity-based person identification using biometric disentanglement

Abstract

A system and method for person identification from video data by disentangling biometric identity features from non-biometric appearance and activity features are disclosed. The system processes RGB video sequences depicting individuals performing various activities to extract spatio-temporal features. These features are separated into distinct biometric identity representations and non-biometric features related to appearance and performed activities. To achieve this separation and minimize appearance bias, the system utilizes an auxiliary supervisory model. At least two implementations of this supervisory model are disclosed: one using semantic supervision via structured embeddings processed through a vision-language model, and another employing silhouette-based feature distillation from a silhouette-trained neural network. Joint training for biometric identification and activity classification ensures accurate identification of individuals independently of facial visibility, clothing differences, or activity variations.