Determining large language model effectiveness utilizing deep learning

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for predicting summary quality scores and determining summary generation costs of large language models to generate a digital document summary. In particular, in one or more embodiments, the disclosed systems extract one or more text segments from a digital document. Further, the disclosed systems generate, utilizing a quality prediction neural network, a predicted summary quality score for each of a plurality of large language models for the one or more text segments. Furthermore, the disclosed systems select a large language model from the plurality of large language models based on the predicted summary quality scores. Moreover, the disclosed systems generate, utilizing the selected large language model, a summary of the digital document.