International Business Machines Corporation

Suzanne L Estrada

Peter Frederick Haggar

Kevin B Haverlock

Christopher S Alkov

Donna Karen Byron

Daniel M Jamrog

Renee F Decker

John A Morganti

Aditya S Gaitonde

Samir J Patel

Joseph F Zaffarano

Timothy P Winkler

Alexander Pikovsky

Robert R Ploger, Iii

Robert R Ploger

Mechanisms are provided for performing load balancing of question processing in a Question and Answer (QA) system, implemented by the data processing system, having a plurality of QA system pipelines. The mechanisms receive an input question for processing by the QA system and determine a predicted question difficulty of the input question. The mechanisms select a QA system pipeline from the plurality of QA system pipelines based on the predicted question difficulty and route the input question to the selected QA system pipeline for processing. In addition, the mechanisms process the input question with the selected QA system pipeline to generate an answer for the input question.

Dynamic load balancing based on question difficulty

A mechanism is provided in a data processing system for rating difficulty of a question. The mechanism receives an input question and generates one or more candidate answers from a corpus of knowledge using a pipeline of software engines. The pipeline of software engines generates a plurality of features extracted from the question, the one or more candidate answers, or the corpus of knowledge. The mechanism then generates a question difficulty score based on the plurality of features using a machine learning model. The machine learning model maps features to assigned weights for scaling the difficulty score.

Rating difficulty of questions

Mechanisms for automatically generating a set of test questions for use in generating a test to be administered to human test takers are provided. The mechanisms ingest a document from a corpus of documents to select a potential test question answer from the document and generate a candidate test question based on the potential test question answer. The mechanisms evaluate the candidate test question using a Question and Answer (QA) system to generate a result indicative of whether the candidate test question should be maintained for test generation. The mechanisms store the candidate test question in the set of test questions in response to a result indicating that the candidate test question should be maintained for test generation.

Dynamic identification and validation of test questions from a corpus

Mechanisms are provided for performing an operation based on an identification of similar lines of questioning by input question sources. Question information identifying extracted features of an input question and a first source of the input question is obtained. A clustering operation is performed to cluster the input question with one or more other questions of a cluster based on a similarity of the extracted features of the input question to features of the one or more other questions. An operation is performed based on results of the clustering of the input question with the one or more other questions.

Categorizing users based on similarity of posed questions, answers and supporting evidence

Aspects include creating a profile for a user from user account data and a user activity log, searching an indexed repository for keywords matching elements from the user account data and the user activity log, and retrieving documents from the indexed repository that are identified based on the searching. Aspects further include creating a custom portal for the user that includes the documents.

Dynamic document collection and custom portal creation

Aspects of processing misaligned annotations include receiving a tokenized document and offset annotation file at a processor. The tokenized document includes a source document and corresponding tokens resulting from a low-level segmentation process. Annotations from the annotation file are applied, in conjunction with tokenization rules, to the source document, and a misalignment responsive to the applying is determined. If the misalignment is caused by an offset mismatch, an offset number of characters between the position counts in the annotation file and the source document is calculated, and the position count in the annotation file is adjusted to coincide with the position count in the source document. If the misalignment is not caused by an offset mismatch, a current position count in the source document is reset to a position count of a previous location in which a most recent alignment between the annotation file and the source document was ascertained.

Misaligned annotation processing

Mechanisms for generating an answer to an input question are provided. The mechanisms receive an input question and route the input question to a plurality of Question and Answer (QA) system pipelines. The mechanisms receive results of processing the input question from each QA system pipeline in the plurality of QA system pipelines. The results comprise one or more candidate answers to the input question and associated confidence scores. The mechanisms adjust, for each QA system pipeline, the confidence scores of the one or more candidate answers associated with the QA system pipeline based on one or more weighting factors. In addition, the mechanisms generate the answer to the input question based on the adjusted confidence scores of the one or more candidate answers for each QA system pipeline.

Generating an answer from multiple pipelines using clustering

Mechanisms for selecting a pipeline of a question and answer (QA) system to process an input question are provided. An input question is received and analyzed to identify at least one feature of the input question. Clustering of the input question, with one or more previously generated clusters of questions, is performed based on the at least one feature of the input question. Based on results of the clustering, a matching cluster, of the one or more previously generated clusters, is identified with which the input question is associated. A QA system pipeline associated with the matching cluster is identified and the input question is processed using the identified QA system pipeline to generate one or more candidate answers for the input question. Each cluster in the one or more previously generated clusters has an associated QA system pipeline.

Growing community of inventors

Boca Raton, FL, United States of America

Suzanne L Estrada