Fakecatcher: detection of synthetic portrait videos using biological signals

Abstract

Detection of synthetic content in portrait videos, e.g., deep fakes, is achieved. Detectors blindly utilizing deep learning are not effective in catching fake content, as generative models produce realistic results. However, biological signals hidden in portrait videos which are neither spatially nor temporally preserved in fake content, can be used as implicit descriptors of authenticity. 99.39% accuracy in pairwise separation is achieved. A generalized classifier for fake content is formulated by analyzing signal transformations and corresponding feature sets. Signal maps are generated, and a CNN employed to improve the classifier for detecting synthetic content. Evaluation on several datasets produced superior detection rates against baselines, independent of the source generator, or properties of available fake content. Experiments and evaluations include signals from various facial regions, under image distortions, with varying segment durations, from different generators, against unseen datasets, and under several dimensionality reduction techniques.