Genetic algorithm testing of application

Abstract

Genetic algorithm testing of a target application includes sending attack utterances to a target application. Genetic algorithm testing further includes receiving, from the target application, application LLM responses generated by the target application and the application LLM responsive to the attack utterances. The application LLM responses are each linked to a corresponding attack utterance to form multiple pairs. Genetic algorithm testing further includes sending, to an evaluation LLM, at least one evaluation LLM prompt including the pairs, identifying, from the set of evaluation results from the evaluation LLM, at least two attack utterances that caused the target application to fail. Genetic algorithm testing further includes mutating the at least two attack utterances to generate a new attack utterance and testing the target application with the new attack utterance to update the set of evaluation results. Genetic algorithm testing further includes presenting the set of evaluation results.