A Comparative Study of AI-Based and Human Ratings of Iranian EFL Learners’ Writing Performance

The rapid development of artificial intelligence has generated increasing interest in the potential use of large language models in language assessment. In particular, recent advances in natural language processing have raised questions regarding the extent to which AI systems can approximate human judgment in the evaluation of second language (L۲) writing and EFL learning. The present study aimed to investigate the relationship between AI-generated scores and human ratings in the assessment of IELTS Task ۲ essays written by Iranian learners of English as a foreign language (EFL). The study focused on the relationship between scores assigned by ChatGPT and those assigned by two human rater. To address this question, a corpus of ۱۰۰ IELTS Task ۲ essays written by Iranian EFL learners was evaluated independently by two experienced human rater as well as by a large language model, ChatGPT. All essays were scored according to the IELTS writing assessment criteria, including task response, coherence and cohesion, lexical resource, and grammatical range and accuracy. The resulting scores were analyzed using descriptive statistics, Pearson correlation analyses, and paired-samples t-tests in order to examine the degree of alignment between AI-generated evaluation and human judgments. Overall, the results of the study provide evidence that large language models have considerable potential as supportive tools in writing assessment. Consequently, AI-based scoring systems may be most effectively utilized as complementary tools that support, rather than replace, human judgment in the evaluation of L۲ writing.

کلیدواژه ها:

Automated Writing Evaluation ، ChatGPT ، IELTS Writing Task ۲ ، Rubric based Scoring ، Writing assessment