Intelligent detection of bias within an artificial intelligence model

Abstract

A system and method includes receiving input and an identification, calculating a bias direction with respect to the identification for an embedding model used by an initial model, and calculating protected gradient scores for tokens in the input. The protected gradient scores are calculated by determining directional derivatives based on model output and bias direction to generate protected gradients, and calculating dot products of the protected gradients and bias direction. The token-level protected gradient scores are aggregated to form an input-level protected gradient score, which is used to determine a fairness indicator. If the fairness indicator is below a threshold, deployment of the initial model is prevented and the model is retrained or fine-tuned using training data selected based on the fairness indicator and identification to produce a trained model with reduced bias. The training data may be selected by removing biased data or synthetically generating new unbiased data.