Document image analysis using joint prediction of layout elements and reading order

Abstract

A labeled data set which indicates layout elements within document images, as well as a reading order for the layout elements, is obtained. Using the data set and a combined loss function, a machine learning model is trained to predict respective content classes of layout elements identified within document images, and reading orders for the layout elements. The combined loss function aggregates losses associated with prediction of the content classes and the reading order. In response to an inference request indicating a document image, an indication of a predicted reading order for layout elements identified in the document image is generated using the trained model.