Prediction of table column items in unstructured documents using a hybrid model

Abstract

One example method includes collecting annotated unstructured documents that each include a table with words whose respective column indices are known, using the documents to train a model to detect a table header in a given document, identifying, by the model, a region of a document that corresponds to a table header in a new document that is not part of the training data, using an algorithm to perform a segmentation process on the table header that identifies column boundaries in the table header, and to use the identified column boundaries to preliminarily assign a respective column index to each word in the table header. Finally, a graph neural network model is run on a graph that includes the words in the table, and running the graph neural network generates a refined prediction of a respective column index for each of the words in the table of the new document.