Abbyy Development Inc.

Abbyy Production LLC

Samsung Electronics Co., Ltd.

Photochemistry Center of Russian Academy of Sciences

Stanislav Semenov

Ivan Germanovich Zagaynov

Aleksandr Valiukov

Mikhail Lanin

Vasily Vasilyevich Loginov

Sergei Nikolaevich Ivanov

Mikhail Mikhailovich Krayushkin

Vladimir Nikolaevich Yarovenko

Yury Petrovich Strokach

Valery Alexandrovich Barachevsky

Aleksey Misyurev

Muhammed Lastanbievich Keshtov

Vasily Loginov

Yury Aleksandrovich Pyankov

Alexandra Antonova

Mikhail Vladimirovich Alfimov

Alexander Alexandrovich Dunaev

Valery Alexandrovich Vasnev

Igor Viktorovich Zavarzin

Andrei Zyuzin

Aspects of the disclosure provide for mechanisms for identification of text fields in documents using neural networks. A method of the disclosure includes obtaining vectors, representative of objects in a document and processing the vectors to generate key hypotheses associating key(s) with one or more objects and value hypotheses associating value(s) with zero or more objects. The method further includes generating key-value association (KVA) hypotheses associating a selected key hypothesis with a selected value hypothesis and characterized by a KVA likelihood score that is based on at least a key likelihood score associated with the selected key hypothesis and a value likelihood score associated with the selected value hypothesis. The method further includes identifying one or more target KVAs of the document using the KVA likelihood scores of the generated KVA hypotheses.

Identification of key-value associations in documents using neural networks

A method of detecting fields in document images includes: receiving a codebook comprising a set of visual words, each visual word corresponding to a center of a cluster of local descriptors; calculating, based on a set of user labeled document images, for each visual word of the codebook, a respective frequency distribution of a field position of a specified labeled field with respect to the visual word; loading a document image for extraction of target fields; calculating a statistical predicate of a possible position of a target field in the document image based on the frequency distributions; and detecting, using the trained model, fields in the document image based on the calculated statistical predicate.

Detecting fields in document images

An example method of extracting structured information from document images comprises: receiving a document image; detecting a tabular structure within the document image; identifying a plurality of rows of the tabular structure, wherein each row of the plurality of rows comprises one or more lines; for each row of the plurality of rows, identifying a set of field types of one or more fields comprised by each line of the one or more lines comprised by the respective row; detecting, in each line of the one or more lines, a set of fields corresponding to a respective set of field types; and extracting information from the set of fields.

Extracting structured information from document images

A computer-implemented method for document clusterization, comprising: receiving an input document; determining, by evaluating a document similarity function, a plurality of similarity measures, wherein each similarity measure of the plurality of similarity measures reflects a degree of similarity between the input document and a corresponding cluster of documents of a plurality of clusters of documents; based on the plurality of similarity measures, determining that the input document does not belong to any of the clusters of documents of the plurality of clusters of documents; creating a new cluster of documents; and associating the input document with the new cluster of documents.

Document clusterization

Mechanisms for document processing and analysis can include receiving a document and identifying, in a data structure, a record corresponding to the document. The record can include one or more entries, where each entry contains data reflecting a respective item of information extracted from a corresponding part of the document. The mechanisms can include determining for each entry of the record, a corresponding degree of association between the entry and a respective item of information referenced by the entry. They can further include updating the corresponding degrees of association, and selecting, among the corresponding degrees of association, a set of corresponding degrees of association whose aggregate degree of association satisfies a criterion.

Extracting information from documents using automatic markup based on historical data

A document processing method includes receiving one or more documents, performing optical character recognition on the one or more documents to detect words comprising symbols in the one or more documents, and determining a encoding value for each of the symbols. It further includes applying a first hash function to each encoding value to generate a first set of hashed symbol values, applying a second hash function to each hashed symbol value to generate a vector array including a second set of hashed symbol values, and applying a linear transformation to each value of the second set of hashed symbol values of the vector array. The method also includes applying an irreversible non-linear activation function to the vector array to obtain abstract values associated with the symbols and saving the abstract values to train a neural network to detect fields in an input document.

Continuous learning for document processing and analysis

A document processing method includes receiving one or more sets of documents, and assigning each document to one or more basic clusters based on the metadata of the document. It further includes for each cluster, training a respective basic cluster model detecting one or more visual element types, and responsive to a first threshold criterion measure related to the one or more basic clusters being satisfied, generating one or more superclusters based on an attribute shared by documents comprised by the plurality of basic clusters. The method also includes training a respective supercluster model detecting the one or more element types and generating a generalized cluster from the one or more superclusters. It includes training a generalized model for the generalized cluster, receiving an input document, assigning the input document to corresponding clusters, and detecting visual elements by processing the input document by each of the corresponding models.

A method of generating and optimizing a codebooks for document analysis comprises: receiving a first set of document images; extracting a plurality of keypoint regions from each document image of the first set of document images; calculating local descriptors for each keypoint region of the extracted keypoint regions; clustering the local descriptors such that each center of a cluster of local descriptors corresponds to a respective visual word; generating a codebook containing a set of visual words; and optimizing the codebook by maximizing mutual information (MI) between a target field of a second set of document images and at least one visual word of the set of visual words.

Optimization and use of codebooks for document analysis

Systems and methods are disclosed to receive a training data set comprising a plurality of document images, wherein each document image of the plurality of document images is associated with respective metadata identifying a document field containing a variable text; generate, by processing the plurality of document images, a first heat map represented by a data structure comprising a plurality of heat map elements corresponding to a plurality of document image pixels, wherein each heat map element stores a counter of a number of document images in which the document field contains a document image pixel associated with the heat map element; receive an input document image; and identify, within the input document image, a candidate region comprising the document field, wherein the candidate region comprises a plurality of input document image pixels corresponding to heat map elements satisfying a threshold condition.

Methods and systems of field detection in a document

An example method of document classification comprises: detecting a set of keypoints in an input image; generating a set of keypoint vectors, wherein each keypoint vector of the set of keypoint vectors is associated with a corresponding keypoint of the set of keypoints; extracting a feature map from the input image; producing a combination of the set of keypoint vectors with the feature map; transforming the combination into a set of keypoint mapping vectors according to a predefined mapping scheme; estimating, based on the set of keypoint mapping vectors, a plurality of importance factors associated with the set of keypoints; and classifying the input image based on the set of keypoints and the plurality of importance factors.

Document clusterization using neural networks

Aspects of the disclosure provide for mechanisms for identification of fields in documents using neural networks. A method of the disclosure includes obtaining a layout of a document, the document having a plurality of fields, identifying the document, based on the layout, as belonging to a first type of documents of a plurality of identified types of documents, identifying a plurality of symbol sequences of the document, and processing, by a processing device, the plurality of symbol sequences of the document using a first neural network associated with the first type of documents to determine an association of a first field of the plurality of fields with a first symbol sequence of the plurality of symbol sequences of the document.

Identification of fields in documents with neural networks without templates

Aspects of the disclosure provide for mechanisms for identification of table partitions in documents using neural networks. A method of the disclosure includes obtaining a plurality of symbol sequences of a document having at least one table, determining a plurality of vectors representative of symbol sequences having at least one alphanumeric character or a table graphics element, processing the plurality of vectors using a first neural network to obtain a plurality of recalculated vectors, determining an association between a first recalculated vector and a second recalculated vector, wherein the first recalculated vector is representative of an alphanumeric sequence and the second recalculated vector is associated with a table partition, and determining, based on the association between the first recalculated vector and the second recalculated vector, an association between the alphanumeric sequence and the table partition.

Identification of table partitions in documents with neural networks using global document context

Aspects of the disclosure provide for mechanisms for identification of blocks of associated words in documents using neural networks. A method of the disclosure includes obtaining a plurality of words of a document, the document having a first block of associated words, determining a plurality of vectors representative of the plurality of words, processing the plurality of vectors using a first neural network to obtain a plurality of recalculated vectors having values based on the plurality of vectors, determining a plurality of association values corresponding to a connections between at least two words of the document, and identifying, using the plurality of recalculated vectors and the plurality of association values, the first block of associated symbol sequences.

Identification of blocks of associated words in documents with complex structures

Mechanisms for identification of text fields in documents using neural networks are described. Identification of text fields includes obtaining a plurality of symbol sequences of a document having a plurality of text fields, determining a plurality of vectors representative of one of the plurality of symbol sequences, processing the plurality of vectors using a first neural network to obtain, based on values of the plurality of vectors, a plurality of recalculated vectors, determining an association between a first recalculated vector of the plurality of recalculated vectors and a first text field of the plurality of text fields, the first recalculated vector being representative of a first symbol sequence of the plurality of symbol sequences, and determining, based on the association between the first recalculated vector and the first text field, an association between the first symbol sequence and the first text field.

Growing community of inventors

Moscow, Russia

Stanislav Semenov