Educational Testing Service

Nitin Madnani

Michael Heilman

Klaus Zechner

Lei Chen

Derrick Higgins

Joel Tetreault

Jill C Burstein

Martin S Chodorow

Beata Beigman Klebanov

Anastassia Loukina

John Sabatini

Adam Faulkner

Jennifer Lentini

Reading comprehension of a user can be assessed by presenting, in a graphical user interface, sequential reading text comprising a plurality of passages. The graphical user interface can alternate between (i) automatically advancing through passages of the reading text and (ii) manually advancing through passages of the reading text within the graphical user interface which is in response to user-generated input received via the graphical user interface. An audio narration is provided during the automatic advancing of the reading text. An audio file is recorded during the manual advancing of the reading text which is used to automatically determine an estimated level of reading comprehension of the user. Data characterizing the determined level of reading comprehension of the user can then be provided (e.g., displayed, loaded into memory, stored on a hard drive, transmitted to a remote computing system, etc.). Related apparatus, systems, techniques and articles are also described.

E-book reading comprehension analytics

Systems and methods are provided for automatically scoring a constructed response. The constructed response is processed to generate a plurality of numerical vectors that is representative of the constructed response. A model is applied to the plurality of numerical vectors. The model includes an input layer configured to receive the plurality of numerical vectors, the input layer being connected to a following layer of the model via a first plurality of connections. Each of the connections has a first weight. An intermediate layer of nodes is configured to receive inputs from an immediately-preceding layer of the model via a second plurality of connections, each of the connections having a second weight. An output layer is connected to the intermediate layer via a third plurality of connections, each of the connections having a third weight. The output layer is configured to generate a score for the constructed response.

Deep convolutional neural networks for automated scoring of constructed responses

Systems and methods are provided for the detection of sentiment in writing. A plurality of texts is received from a larger collection of writing samples with a computer system. A set of seed words from the plurality of texts are labeled as being of positive sentiment or of negative sentiment with the computer system. The set of seed words is expanded in size with the computer system to provide an expanded set of seed words. Intensity values are assigned to words of the expanded set of seed words. Each of the words of the expanded set of seed words is assigned three intensity values: a value corresponding to the strength of the word's association with a positive polarity class, a value corresponding to the strength of the word's association with a negative polarity class, and a value corresponding to the strength of the word's association with a neutral polarity class.

Computer-implemented systems and methods for detection of sentiment in writing

In accordance with the teachings described herein, systems and methods are provided for measuring a user's comprehension of subject matter of a text. A summary generated by the user is received, where the summary summarizes the text. The summary is processed to determine a first numerical measure indicative of a similarity between the summary and a reference summary. The summary is processed to determine a second numerical measure indicative of a degree to which a single sentence of the summary summarizes an entirety of the text. The summary is processed to determine a third numerical measure indicative of a degree of copying in the summary of multi-word sequences present in the text. A numerical model is applied to the first numerical measure, the second numerical measure and the third numerical measure to determine a score for the summary indicative of the user's comprehension of the subject matter of the text.

System and method for automated scoring of a summary-writing task

Systems and methods are provided for correcting a grammatical error in a text sequence. A first text sequence in a first language is received. The first text sequence is translated to a second language to provide a first translated text. The first text sequence is translated to a third language to provide a second translated text. The third language is different from the second language. The first translated text is translated to the first language to provide a first back translation. The second translated text is translated to the first language to provide a second back translation. A plurality of candidate text sequences that include features of the first back translation and the second back translation are determined. The plurality of candidate text sequences include alternative grammatical options for the first text sequence. The plurality of candidate text sequences are scored with the processing system.

Round-trip translation for automated grammatical error correction

In accordance with the teachings described herein, systems and methods are provided for identifying organizational elements in argumentative or persuasive discourse. A text that has been annotated is received. The annotated text includes argumentative or persuasive discourse that includes claims and evidence and organizational elements configured to organize the claims and evidence. Annotations of the annotated text distinguish the organizational elements from the claims and evidence. A rule set or a feature set is identified from the annotated text, where the rule set or the feature set includes textual patterns or word frequency features related to the organizational elements of the annotated text. A model is built based on the annotations and on the rule set or the feature set. The model is configured to identify organizational elements in a new text. The model is applied to the new text.

Growing community of inventors

Princeton, NJ, United States of America

Nitin Madnani