Retrieval-augmented generation (rag) system optimization

Abstract

A method includes obtaining training data for a retrieval-augmented generation (RAG) architecture having retriever and generative models. The retriever model is configured to identify information chunks relevant to input queries, and the generative model is configured to generate outputs based on the information chunks and the input queries. The method also includes generating a prompt for the generative model and generating multiple sets of queries for the retriever model. Each query in the multiple sets of queries is configured to cause the retriever model to select a set of information chunks associated with the prompt. The method further includes generating multiple responses to the prompt using the generative model and the sets of information chunks and determining rewards associated with the RAG architecture based on the responses. In addition, the method includes training the generative model based on the training data and the rewards to produce an updated RAG architecture.