International Business Machines Corporation

Paul Joseph Hake

Andrew Jason Lavery

Igor S Ramos

Scott Robert Carrier

Faheem Altaf

Aspects include receiving a document and classifying at least a subset of the document as having a first type of data. Features are extracted from the document. The extracting includes initiating processing of the at least a subset of the document by a first processing engine that was previously trained to extract features from the first type of data. The extracting also includes initiating processing of a remaining portion of the document not included in the at least a subset of the document by a second processing engine that was previously trained to extract features from a second type of data. The first type of data is different than the second type of data. Features are received from one or both of the first processing engine and the second processing engine. The received features are stored as features of the document.

Handling form data errors arising from natural language processing

Aspects of the invention include a computer-implemented method for generating promise identifiers for documents. Aspects include processing a document including a reference, wherein processing includes performing natural language processing (NLP) the document, and identifying the reference included in the document. Aspects also include generating a promise identifier for the reference in the document, and responsive to processing the document, resolving the promise identifier for the reference by providing data of the reference associated with the promise identifier. Aspects of the invention also include a computer program product and system for generating promise identifiers for documents.

Promised natural language processing annotations

Aspects of the invention include resolving future reference identifiers for documents. Aspects of the invention include processing a document including a reference to a future event, wherein processing includes performing natural language processing (NLP) on the document, and identifying the reference to the future event included in the document. Aspects of the invention also include generating a future reference identifier for the reference to the future event, and responsive to processing an occurrence of the future event, resolving the future reference identifier by providing data from a subsequent document for the future event associated with the future reference identifier.

Future potential natural language processing annotations

Aspects of the invention include detecting and processing sections spanning processed document partitions by caching a document partition. The document partition includes metadata indicating that the document partition is a portion of a whole document. Aspects also include pairing a candidate paragraph from the document partition with a cached paragraph segment and determining, using a coherence model, a probability that the candidate paragraph and the cached paragraph segment constitute a semantically coherent paragraph. Aspects further include discarding the cached paragraph segment and processing the candidate paragraph and the cached paragraph segment separately based on a determination that the probability is less than a threshold level and processing the candidate paragraph and the cached paragraph segment together as a cross-partition paragraph based on a determination that the probability is greater than the threshold level.

Detecting and processing sections spanning processed document partitions

According to one or more embodiments of the present invention, an input request to a natural language processing (NLP) system is optimized. A window-size is selected for annotating an input corpus. The corpus is divided into partitions of the window-size, each partition processed separately. Further, a first set of entities is identified in a first partition, and a second set of entities in a second partition. Further, a third partition containing a first segment and a second segment is determined. The first segment overlaps the first partition, and the second segment overlaps the second partition. The method further includes identifying a third set of entities in the third partition. In response to the third set of entities being distinct from a set of entities from the first segment and the second segment, the window-size is adjusted. The input request for the NLP system is generated using the adjusted window-size.

Sliding window to detect entities in corpus using natural language processing

A method, computer system, and computer program product for document superposition detection and correction are provided. The embodiment may include receiving a plurality of scanned documents. The embodiment may also include analyzing the plurality of the scanned documents. The embodiment may further include determining if there is a superposition scanning defect within the scanned documents based on the analysis. The embodiment may also include in response to the plurality of the scanned documents containing the superposition scanning defect, identifying a subset of documents within the plurality of scanned documents containing the determined superposition scanning defect. The embodiment may further include transmitting a notification to a user, wherein the notification identifies the subset and that the subset requires a rescan.

Growing community of inventors

Madison, CT, United States of America

Paul Joseph Hake