Generating a unified metadata graph via a retrieval-augmented generation (rag) framework systems and methods

Abstract

Systems and methods for reducing data retrieval times when accessing siloed data across disparate locations by generating a unified metadata graph via a Retrieval-Augmented Generation (RAG) framework are disclosed. The system selects a first Large Language Model (LLM) prompt corresponding to a first metadata identifier of the set of metadata identifiers. The system then augments the first LLM prompt with the first metadata identifier to be provided to an LLM, where the LLM is configured to generate a first intermediate output. The system then augments the first LLM prompt with the second set of metadata identifiers corresponding to the first metadata identifier to be provided to the LLM, where the LLM generates a second intermediate output. The system then generates a domain-specific unified metadata graph. Responsive to determining that first performance metrics fail to satisfy a performance criteria, the system performs an update process on the domain-specific unified metadata graph.