Dual-capacity machine learning based validation framework

Abstract

A method for automatically evaluating code using a dual-capacity machine learning based validation framework. A target code generated by a code generator and ground truth code are received by the validation framework. A large language model is prompted, by the validation framework, with the target code, the ground truth code, and instructions to evaluate the target code by performing operations. The operations include segmenting the target code into a plurality of target code chunks and the ground truth code into a plurality of ground truth code chunks and mapping each target code chunk to corresponding ground truth code chunks. The operations also include evaluating each mapped target code chunk based on the corresponding ground truth code chunk to yield a plurality of scores. The operations also include determining a metric for the target code based on the plurality of scores.