Deep learning based document image embeddings for layout classification and retrieval

Abstract

Image documents that have a visually perceptible geometric structure and a plurality of visually perceptible key-value pairs are grouped. The image documents are processed to generate a corresponding textually encoded document. The textually encoded documents are each assigned into one of a plurality of layout groups, wherein all textually encoded documents in a particular layout group share a visually perceptible layout that is substantially similar. Triplets are selected from the layout groups, where two documents are from the same layout group and one document is from a different layout group. The triplets are processed with a convolutional neural network to generate a trained neural network that may be used to classify documents in a production environment such that a template designed on one image document in a group permits an extraction engine to extract all relevant fields on all image documents within the group.