International Business Machines Corporation

Michael W Schroeder

Andrew Ronald Freed

Joel C Dubbels

Elizabeth T Dettman

Michael Terry Payne

Corville O Allen

John Edward Petri

Mark Gregory Megerian

Frances E Stewart

Embodiments presented herein disclose techniques for transforming input documents having disparate formats into a normalized format (e.g., Atom, RSS, HTML, customized XML, etc.). According to one embodiment, a plurality of fields is identified in an input document that has a given format. Each field includes a descriptor and text content associated with the descriptor. For each field, semantic properties are evaluated for the descriptor and text content against a plurality of mapping rules to determine whether the field is consistent with one of a plurality of fields of a target format. Each mapping rule specifies characteristics associated with one of the fields in the target format. Once so determined, a mapping from the first field to the second field is defined.

Natural language processing&#x2014;assisted extract, transform, and load techniques

Determining the likelihood that an input descriptor and associated text content match a target field using natural language processing techniques in preparation for an extract, transform and load process

Mechanisms are provided, in a Question and Answer (QA) system comprising a processor and a memory, for debugging code. An input question identifying an error during execution of code is processed by the QA system using a corpus corresponding to a software product associated with the code, thereby generating a first candidate answer set. The QA system processes the input question using a code change record repository identifying changes to the code performed over time to generate a second candidate answer set. The QA system generates a final answer to the input question based on the first and second candidate answer sets and outputs the final answer to the input question. The final answer to the input question identifies at least one of a source, in the code, of the error or a solution to resolving the error.

Debugging code using a question and answer system based on documentation and code change records

An approach is provided in which a knowledge manager analyzes multiple document phrases using a natural language processing model and generates multiple interpretations based upon the analysis. The knowledge manager identifies misinterpretation patterns by comparing the multiple interpretations with multiple corrections that include corrections to the multiple interpretations. In turn, the knowledge manager generates interpretation rules based upon the identified patterns and applies the interpretation rules to the natural language processing model.

Pattern identification and correction of document misinterpretations in a natural language processing system

A mechanism is provided in a data processing system for partial ingestion of content. The mechanism receives new content to be ingested into a corpus of information. The mechanism applies a plurality of sub-pipelines of annotation engines against the new content in order of effectiveness. The plurality of sub-pipelines include all annotation engines of an ingestion pipeline. Each sub-pipeline within the plurality of sub-pipelines generates one or more intermediate output objects. The mechanism provides access to the one or more intermediate output objects.

Real-time or frequent ingestion by running pipeline in order of effectiveness

A mechanism is provided, in a data processing system comprising a processor and a memory configured to implement a question and answer (QA) system, for effectively ingesting data for answering questions in the QA system. A received input question having a set of question characteristics is parsed, which are compared to question characteristics associated with a set of previous questions. Responsive to the set of question characteristics matching the question characteristics associated with one or more previous questions above a related-question predetermined threshold, identification is made as to whether answers to the one or more previous questions were obtained from static information sources or real-time information sources. Responsive to the answers to the one or more previous questions being obtained from the real-time information sources above the predetermined real-time threshold, real-time information sources related to the characteristics of the input question are initially utilized to answer the input question.

Effective ingesting data used for answering questions in a question and answer (QA) system

System and computer program product to perform an operation for query processing based on normalized search terms. The operation begins by, responsive to receiving a query, generating a normalized search term for a concept in the query based on a first language model, of a plurality of language models each having a predefined association with a respective concept. The operation then modifies the query to include the normalized search term, and executes the modified query against an indexed corpus of evidence including a first item of evidence. The operation then, upon determining that the first item of evidence includes the normalized search term, returns the first item of evidence as responsive to the query.

Growing community of inventors

Rochester, MN, United States of America

Michael W Schroeder