Systems and methods for content scoring of spoken responses

Abstract

Computer-implemented systems and methods are provided for automatically scoring the content of moderately predictable responses. For example, a computer performing the content scoring analysis can receive a response (either in text or spoken form) to a prompt. The computer can determine the content correctness of the response by analyzing one or more content features. One of the content features is analyzed by applying one or more regular expressions, determined based on training responses associated with the prompt. Another content feature is analyzed by applying one or more context free grammars, determined based on training responses associated with the prompt. Another content feature is analyzed by applying a keyword list, determined based on the test prompt eliciting the response and/or stimulus material. Another content feature is analyzed by applying one or more probabilistic n-gram models, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a POS response vector, determined based on the response, to one or more POS training vectors, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a response n-gram count to one or more training n-gram counts using an n-gram matching evaluation metric (e.g., BLEU). Another content feature is analyzed by comparing the response to one to training responses associated with the prompt using a dissimilarity metric (e.g., edit distance and word error rate).