Unsupervised validation framework for large language model (llm) outputs

Abstract

The disclosure relates to a system and method for unsupervised validation of Large Language Model (LLM)-generated outputs. The method includes the steps of receiving input data, which may comprise structured or unstructured text; generating an LLM output based on the input data; extracting a first set of topics from the input data, with each topic represented by a set of keywords; converting the first set of topics into human-readable reference data; extracting a second set of topics from the LLM output; and converting the second set of topics into human-readable candidate data. The method further includes comparing the reference data with the candidate data using one or more performance metrics. Finally, the method determines a validation score based on the comparison results. The disclosed method allows for the objective validation of LLM-generated content, enhancing accuracy and efficiency in diverse applications such as customer service, technical documentation, and legal analysis.