Factual-consistency detection using chunking and a natural language interference model

Abstract

Generative AI models exhibit remarkable potential. However, hallucinations across various tasks present a significant challenge, particularly for long inputs that current approaches struggle to address effectively. The present disclosure sets forth a factual-consistency detection method that can be used to determine if AI-generated text is factually consistent with a reference document. The method employs a novel chunking mechanism in which the reference document is divided into larger chunks (i.e., more than one sentence) and the AI-generated text is evaluated against each chunk for factual consistency. The use of chunks larger than a sentence improves both accuracy and efficiency. Additionally, a novel search tree approach that leverages chunking enables the system to provide support for a factual consistency decision by identifying the most relevant sentence(s) in the reference document while using fewer model runs than previous approaches.