Determination of the decision-relevant image components for an image classifier by targeted modification in the latent space

Abstract

A method for measuring components of an input image on which an image classifier bases its decision regarding the assignment of the input image to class(es) of a predefined classification. The method includes: processing the input image by the image classifier into an intermediate product; mapping the intermediate product on a classification score with respect to at least one target class; ascertaining a perturbation from counter image(s) which is/are preferentially assigned by the image classifier to at least one class other than the target class; providing at least one binary mask; creating at least one modification, in which pixels established by the binary mask are replaced with pixels of the perturbation corresponding thereto; mapping the modification on a classification score with respect to a predefined class; and ascertaining from the classification score to what extent the binary mask indicates the sought-after decision-relevant components of the input image.