International Business Machines Corporation

Anna Chaney

Stefan Van Der Stockt

Alexander Block

Kai G H Young

John Michael Ganci, Jr

Craig M Trim

Methods, computer program products, and systems are presented. The methods include, for instance: obtaining an input text for an output speech. The number of words and syllables are counted in each sentence, and a mean sentence length of the input text is calculated. Each sentence length is checked against the mean sentence length and a variation for each sentence is calculated. For the input text, the consumability-readability score is produced as an average of variations for all sentences in the input text. The consumability-readability score indicates the level of satisfaction for the listener of the output speech based on the input text.

Assessment of speech consumability by text analysis

A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface.

Automatically assessing question answering system performance across possible confidence values

A mechanism is provided in a data processing system for comparing similarity of questioning answer system ground truth answers to question answering system result answers. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each result answer in the question answering system results to a corresponding ground truth answer. The matching operation is an asymmetric containment comparison and wherein the matching operation determines similarity of each result answer to the corresponding ground truth answer. The mechanism determines whether each result answer is right or wrong based on results of the matching operation.

Growing community of inventors

Austin, TX, United States of America

Anna Chaney