Systems and methods for hybrid lexical-vector retrieval in retrieval-augmented generation models

Abstract

Systems and methods perform hybrid lexical-vector retrieval in a retrieval-augmented generation (RAG) framework. A user query is pre-processed using named-entity recognition, date standardization, and optional out-of-domain detection. The system performs both lexical search and semantic vector similarity search over a corpus of document chunks, generating ranked candidate sets that are merged by elevating overlapping results and interleaving remaining items according to a predetermined rule. A top subset of chunks is selected based on the combined ranking and provided to a large language model (LLM), together with intent-specific instructions determined through query-classification logic. The LLM generates an answer grounded in the retrieved material and formatted according to a standardized template associated with the detected intent.