Language model hallucination mitigation using contrastive decoding

Abstract

Devices and techniques are generally described to mitigate hallucination for language models (LMs) using contrastive decoding. In various examples, first context data and first adversarial data may be determined based on a natural language input. First prompt including the natural language input, second prompt including the natural language input and the first context data, and third prompt including the natural language input and the first adversarial data may be generated. An LM may generate first vector of logits for the first prompt, second vector of logits for the second prompt, and third vector of logits for the third prompt. A decoder may generate first output data based on a combination of the first vector of logits, the second vector of logits, and the third vector of logits.