Deepfake detection using synchronous observations of machine learning residuals

Abstract

Systems, methods, and other embodiments associated with computer deepfake detection are described. In one embodiment, a method includes converting audio-visual content of a person delivering a speech into a set of time series signals. Residual time series signals of residuals that indicate an extent to which the time series signals differ from machine learning estimates of authentic delivery of the speech by the person are generated. Residual values from one synchronous observation of the residual time series signals are placed into an array of residual values for a point in time. A sequential analysis of the residual values of the array is performed to detect an anomaly in the residual values for the point in time. In response to detection of the anomaly, an alert that deepfake content is detected in the audio-visual content is generated.