Efficient scanning for threat detection using in-doc markers

Abstract

The disclosed technology teaches reducing threat detection processing, including recognizing that a file is an edited version of a previously processed file and retrieving, from an archive, metadata values, hashes for property groups and an entropy measure of the previously processed file. Also included is parsing the file into metadata values and property groups and calculating hashes of the property groups and entropy measure for the file. The method further includes applying similarity measures to compare the metadata values, the entropy measures, and the hashes on the property groups, for the edited version and the previously processed file. When any similarity measure or combination of similarity measures reaches a trigger, the technology teaches processing the file by using a threat detection module to detect malware. Property groups include core properties, application properties, document content and programming scripts for the edited version of the file and the previously processed file.