U. S. Bank Corporation

U.S. Bancorp, National Association

Ali Fathi

Samuel A Assefa

Giacomo Domeniconi

Samuel Atkins

The disclosed embodiments describe a method, system, and computer-readable medium for reducing hallucinations in a large language model in the field of natural language processing involving identifying a plurality of input-output pairs, each input-output pair assigned to a probability and comprising an example input text string and a corresponding output text string, sampling a set of input-output pairs from the plurality of input-output pairs based on the probabilities assigned to the plurality of input-output pairs, generating a list of aggregated input-output pairs from the set of input-output pairs, generating one or more queries by iteratively replacing identified variables within each aggregated input-output pair with a different value from a database, and training, using the one or more queries, a first large language model to convert input queries to machine-readable prompts configured for input into a second large language model.

Controlled probabilistic sentence space expansion

A system and method includes receiving input and an identification, calculating a bias direction with respect to the identification for an embedding model used by an initial model, and calculating protected gradient scores for tokens in the input. The protected gradient scores are calculated by determining directional derivatives based on model output and bias direction to generate protected gradients, and calculating dot products of the protected gradients and bias direction. The token-level protected gradient scores are aggregated to form an input-level protected gradient score, which is used to determine a fairness indicator. If the fairness indicator is below a threshold, deployment of the initial model is prevented and the model is retrained or fine-tuned using training data selected based on the fairness indicator and identification to produce a trained model with reduced bias. The training data may be selected by removing biased data or synthetically generating new unbiased data.

Growing community of inventors

San Francisco, CA, United States of America

Ali Fathi