Information extraction system for unstructured documents using retrieval augmentation with custom query parameters

Abstract

A system for extracting a number of data elements from one or more unstructured data sources. The system may separate the text from the tables in a document, such that only the table data may be sent to the large language model (LLM), when the LLM only needs to review the table data. The system may form table chunks and text chunks and index the chunks using a vector embedding. The system may, in response to a request to extract the data elements, retrieve and send the targeted table chunks or text chunks to the LLM to extract the data elements. Retrieval of the chunks may be based on different retrieval parameters including a chunk type designation, a document type designation, a search type designation, and a reach criterion. Hierarchical sets of search criteria may be provided to ensure a minimal number of chunks are retrieved.