International Business Machines Corporation

Christopher Phipps

Charles Evan Beller

Edward Graham Katz

Michael Drzewucki

Paul J Chase, Jr

Richard L Darden

Kristen Maria Summers

Julie T Yu

Sean L Bethard

James E Ramirez

Andrew Ronald Freed

John A Riendeau

Sean Thomas Thatcher

Kyle G Christianson

A method, computer system, and a computer program product for converting a plurality of factoid answers into a plurality of structured relations for storage in a structured knowledge base is provided. The present invention may include receiving a query from a user. The present invention may also include generating a plurality of possible factoid answers. The present invention may then include determining a plurality of confidence scores for the plurality of possible factoid answers. The present invention may then include generating a plurality of certain factoid answers from the plurality of possible factoid answers. The present invention may then include identifying a plurality of special target items. The present invention may further include generating a plurality of structured relations for each certain factoid answer and the identified plurality of special target items. The present include may also include storing the generated plurality of structured relations into the structured knowledge base.

Post-processor for factoid answer conversions into structured relations in a knowledge base

A method, system and computer-usable medium are disclosed for automated analysis of ground truth using confidence model to prioritize correction options. In certain embodiments, the ground truth data is analyzed to identify review-candidates. A confidence level may be assigned to each of the identified review-candidates and the review-candidates are prioritized, at least in part, using the assigned confidence levels. The review-candidates are electronically presented in prioritized order to solicit verification or correction feedback for updating the ground truth data.

System and method for automated analysis of ground truth using confidence model to prioritize correction options

Embodiments relate to an intelligent computer platform to create a document specific error correction model for amending OCR values. An image of a document is received and OCR is applied to the received image. Text is extracted from at least one static content field and the extracted text is compared to stored text from known static content. Responsive to a deviation identified in the comparison, a document specific error correction model is created and leveraged to correct OCR output. The model generates one or more variants for the dynamic content field associated with the static content field having the identified deviation. The generated variant(s) is subject to processing and one of the variants is selected as amended document content. A new document version is created from the amendment, the new document version including corrected OCR output.

Optical character recognition error correction model

Embodiments for finding elements in optical character recognition (OCR) documents are provided. An indication of a selected portion of document is received. Salient pixels in the selected portion of the document are determined. Properties of the salient pixels in the selected portion of the document are identified. The properties of the salient pixels in the selected portion of the document are compared to properties of pixels in each of a plurality of portions of an OCR-converted version of the document. A cognitive analysis is utilized to select at least some of the plurality of portions of the OCR-converted version of the document as suspected matches to the selected portion of the document.

Methods and systems for finding elements in optical character recognition documents

An apparatus comprising a memory and a processor configured for semi-autonomous natural language processing domain adaptation related activities. The processor coupled to the memory and configured to identify a corpus of documents of an evaluation domain and generate a first lexicon based on the corpus of documents of the evaluation domain, and determine a threshold that indicates a sufficiency of domain adaptation of the evaluation domain based at least in part on the first lexicon. The processor is further configured to identify a corpus of documents of a client domain, generate a second lexicon based on the corpus of documents of the client domain, determine a metric associated with the corpus of documents of the client domain and the second lexicon, and determine that domain adaptation of the client domain is complete when the metric exceeds the threshold.

Process for identifying completion of domain adaptation dictionary activities

A mechanism is provided in a data processing system for identifying nonsense passages. An annotator in a natural language processing pipeline configured to execute in the data processing system annotates an input passage in a corpus with linguistic features to form an annotated passage. A domain-specific policy is associated with a domain of the corpus. A metric counters component in the natural language processing pipeline counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The metric counters component of the natural language processing pipeline determines a value for a metric based on the set of feature counts. The metric is specified in the domain-specific policy. A comparator component of the natural language processing pipeline compares the value for the metric to a predetermined model threshold. The threshold is specified in the domain-specific policy. A filter component of the natural language processing pipeline identifies whether the input passage is a nonsense passage based on a result of the comparison.

Identifying nonsense passages in a question answering system based on domain specific policy

Embodiments are directed to a watched questions threshold filtering system that functions to determine and deliver to a user relevant and significant data changes with respect to a user's goals, as defined by a notification threshold value provided by the user. The user is provided with an option to flag one or more queries for automatic re-querying. Confidence scores are processed on new data (i.e., data ingested after the original question was asked) by utilizing a confidence threshold for indicating if the new data warrants alerting a user.

Dynamic threshold filtering for watched questions

A method, computer system, and a computer program product for leveraging coherent question sequences is provided. The present invention may include receiving an initiating question. The present invention may include receiving a subsequent question. The present invention may include determining that the received subsequent question is not a rephrasing of the received initiating question. The present invention may also include determining that the received subsequent question is not beginning a new question topic based on determining that the received subsequent question is not a rephrasing of the received initiating question. The present invention may then include propagating a conversational context based on determining that that received subsequent question is not beginning a new question topic. The present invention may include generating and scoring an answer based on the propagated conversational context. The present invention may lastly include outputting the answer.

Leveraging contextual information in topic coherent question sequences

A grammatically diverse test set of natural language sentences for a deep question answering system is provided by analyzing a given sentence to characterize its syntactical classification, and adding the sentence to the test set if its classification is sufficiently different from other sentences already in the test set. A particular sentence may be selected for inclusion according to a desired syntactic distribution. Multiple sentences having the exact same classification may be allowed subject to a maximum number of such sentences. The test set is adapted to an element of interest by characterizing each syntactical classification relative to the element of interest. The analysis derives a parse tree, identifies a particular node of the tree corresponding to the element of interest, and extracts syntactic information by traversing the tree starting at the particular node and ending at the root node of the tree according to different traversal schemes.

Generation of a grammatically diverse test set for deep question answering systems

A mechanism is provided in a data processing system for identifying nonsense passages. The mechanism annotates an input passage with linguistic features to form an annotated passage. The mechanism counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The mechanism determines a value for a metric based on the set of feature counts and compares the value for the metric to a predetermined model threshold. The mechanism identifies whether the input passage is a nonsense passage based on a result of the comparison.

Post-processing for identifying nonsense passages in a question answering system

Syntactic classification of natural language sentences with respect to a targeted element

Large lists of domain-specific terms are classified as a particular kind of linguistic object, e.g., lexical answer type T versus canonical answer E, based on features from a domain-specific corpus which have been found to distinguish between the linguistic objects. The distinguishing features can be identified in the corpus based on sets of the linguistic objects derived from question-and-answer pairs. A classifier can be trained using the distinguishing features, and the classification carried out using that classifier. The distinguishing features can include one or more syntactic features or one or more lexical features. The linguistic objects (the T and E training sets) can be extracted from the question-and-answer pairs automatically via text analysis if manually curated lists are not available. The classified terms can be included in a domain-specific lexicon which facilitates a deep question answering system to yield an answer to a question.

Growing community of inventors

Arlington, VA, United States of America

Christopher Phipps