Self-supervised retriever optimization via attention-derived feedback in retrieval augmented generation systems

Abstract

Certain aspects of the disclosure provide a method for updating a retrieval augmented generation (RAG) system. The method includes receiving a user query and retrieving a set of datasets from an external knowledge base associated with a language model. The user query and retrieved datasets are provided to the language model as input tokens, which generates a response comprising output tokens. The system then extracts cross-attention weights from the language model, indicating how much attention each output token paid to each input token. Using these weights, the system generates attention scores for each dataset and identifies a top-k set of most attended datasets. If the generated response is determined to be relevant to the user query, the top-k most attended datasets are labeled as positive examples. The system then updates its parameters to prioritize retrieving these positive examples for future queries, enabling continuous self-supervised improvement.