Adversarial machine learning attack detection and prevention system

Abstract

A machine learning (ML) attack detection and prevention system may monitor operation of an explainable artificial intelligence (AI) model processed by an application server to provide a product or service to a user. The AI model outputs explainable data and/or other outputs used in decision making. The ML attack detection and prevention system derives business rules based on the explainable data produced by the explainable AI model. Additionally, the ML attack detection and prevention system processes rules repositories of simulation failure data, historical failure data and business rules to derive possible fraud rules based on the data collected in above step. Based on a comparison of rules derived from the AI model output and the possible fraud rules, the ML attack detection and prevention system issues an alert if a fraud condition is suspected and causes the application server to revert back to a previous version of the AI model.