Hallucination detection and handling for a large language model based domain-specific conversation system

Abstract

Systems and methods are described herein for detecting and reducing hallucinations and improving the reliability of an LLM-based QA system. Particularly, the disclosure provides a set of hallucination detection and handling approaches that address different types of hallucinations of the QA system. The hallucination detection approaches described herein include comparing a natural language answer with context information by way of sentence similarity estimation and keyword matching. The hallucination handling approaches described herein include removing hallucinated sentences from the natural language answer or regenerating the natural language answer using better context information, depending on a level of hallucination detected in the natural language answer. A hybrid framework is also provided that systematically combines the hallucination detection approaches and hallucination handling approaches into one system to achieve an optimal hallucination-reduction performance for the QA system.