Method for accelerated long document search using hilbert curve mapping

Abstract

A system and method includes a textual document search engine. The method initializes the textual document search engine by inputting, into a memory, a very large number of documents, where each document has textual sentences, and each sentence of each document has m semantic embedding vectors. The method maps the m semantic embedding vectors to 1-dimensional vectors of Hilbert numbers using a Hilbert curve transformation, constructs an index table with the 1-dimensional vectors, and stores the index table in the memory. The index table is used for efficient search by inputting a full query document, which has embedding vectors corresponding to sentences in the query document, where the query embedding vectors are mapped into 1-dimensional vectors of Hilbert numbers using Hilbert curve transformation. The index table is searched using the Hilbert numbers and candidate documents are retrieved that are similar to the query document based on the Hilbert numbers.