Educational Testing Service

Claudia Leacock

Martin S Chodorow

Daniel Marcu

Jill C Burstein

Vyacheslav Andreyev

Magdalena Wolska

Eleanor Bolge

Systems and methods for creating a mathematical model for use in identifying discourse elements are described. A plurality of first essays relating to a particular subject are received, where each first essay is in an electronic format. Annotations for each first essay are received, where each annotation identifies at least one discourse element. Features are identified with a processor, where each feature is exhibited by at least one identified discourse element. Empirical frequencies are computed with a processor, where each empirical frequency relates to the presence of a feature with respect to the identified discourse elements across the plurality of first essays. Each empirical frequency is associated with the related identified discourse element with a processor. The empirical frequencies are utilized to select discourse elements in at least one second essay.

Methods for automated essay analysis

A concept rater module is utilized to automatically grade or score constructed responses based on a model answer. The concept rater module may be configured to accept a model answer as input. The model answer may be used as a grading key by the concept rater module. The concept rater module may be further configured to accept student responses in a file format. The file format may be ASCII text, a formatted word processing (e.g., WORDPERFECT, MICROSOFT WORD, etc.) and the like. The concept rater module may be further configured to process a student response into a canonical representation of the student response. The canonical representation of the student response is compared against the model answer by the concept rater module. From the comparison, a score is generated which represents that student's ability to cover all the key concepts.

System for rating constructed responses based on concepts and a model answer

An essay is analyzed automatically by accepting the essay and determining whether each of a predetermined set of features is present or absent in each sentence of the essay. For each sentence in the essay a probability that the sentence is a member of a certain discourse element category is calculated. The probability is based on the determinations of whether each feature in the set of features is present or absent. Furthermore, based on the calculated probabilities, a sentence is chosen as the choice for the discourse element category.

Growing community of inventors

New York, NY, United States of America

Claudia Leacock