A Validation of EFL Essay Assessment Based on Corpus Indices and Error Analysis

Abstract

An ever-growing demand for teaching English writing skills has been impeded by the logistic problem posed by the inherently time-consuming nature of the sophisticated process of writing assessment, whose validity and inter-rater reliability can be deteriorated by idiosyncratic human ratings. Thus, a wide variety of computer-based automatic essay scoring (AES) schemes have been developed and employed to enable language educators to cope with the issue of practicality and subjectivity in the scoring of performance tests. The first part of the present research (pilot study) is intended to probe the validity of AES-based ratings in comparison with human ratings. A comparative analysis of AES-based ratings and human ratings appears to substantiate the robustness of the AES scheme. Based on the positive results of AES, the second part of the research (main study) attempts to capitalize on quantitative analysis of corpus linguistic indices and a systematic qualitative error analysis to explore the validity of human assessment of essays collected from a writing test. The results of data analyses are provided to discuss the possibility of utilizing a corpus-based analysis in order to enhance the validity of the essay assessment process, which will lead to more effective and plausible EFL writing teaching and testing