Efficacy of an Automatic Essay Scoring-based English Writing In-service Training Program

Abstract

With an ever-growing demand for writing instruction in the English Language Teaching (ELT), English teachers are expected to develop teaching skills in English writing by improving their own English writing skills. This need for teaching and learning English writing skills for teachers, however, has been impeded by the logistic problem posed by the inherently time-consuming nature of the evaluation process. Thus, a wide variety of automatic essay scoring (AES) schemes have been developed to empower teachers in coping with this subjective scoring problem. The present study is intended to probe the robustness of AES-based ratings, the efficacy of an in-service training program designed to enable English teachers to develop their English writing skills with the help of an AES-based program, and the overall quality of the teaching program as perceived by participants. A comparative analysis of AES-based ratings and human ratings substantiates the robustness of the AES scheme. Overall exploration of the program’s efficacy reveals that the educational program exerted a positive impact on the subjects’ attitudes toward English writing and their acquisition of mechanical grammar. The survey demonstrates that the majority of respondents were satisfied with the overall quality of the program and the reliability of AES ratings.