Evaluating AI-Driven Feedback in IELTS Writing: A Comparative Analysis of Grok and Qualified Human Examiners

AI tools promise efficient language assessment but face questions about reliability compared to human examiners in high-stakes tests such as the IELTS Writing. To address this, the study evaluated Grok’s feedback efficacy against that of three Iranian IELTS examiners for Tasks ۱ and ۲, focusing on alignment in scoring accuracy and the quality of diagnostic feedback. A mixed-methods design was employed to analyze ۵۰ writing samples, comprising ۲۵ Task ۱ and ۲۵ Task ۲ responses. These samples were scored by Grok in both standard and enhanced modes and compared with human examiner scores, which were awarded based on the IELTS Writing Rubric. Quantitative metrics assessed alignment, while thematic analysis explored qualitative differences. The findings revealed moderate alignment between Grok’s scores and human examiner scores, with Grok performing better in Task ۱ and the enhanced mode. However, it struggled significantly in Task ۲'s vocabulary assessment due to an overreliance on metrics such as the type-token ratio. Persistent issues such as surface-level focus, rhetorical insensitivity, and non-native bias were observed, although these were partially mitigated in the enhanced mode. While Grok demonstrated consistency and excelled in Task ۱ detail, it lacked fairness and diagnostic depth for Task ۲. Consequently, it is suitable for formative feedback but requires human oversight for summative assessments. In light of these findings, the study advocates hybrid models that combine AI efficiency with human judgment, underscoring the indispensable role of human expertise in maintaining fairness and depth in high-stakes test scoring.

کلیدواژه ها:

AI-driven Feedback ، Automated Essay Scoring ، Grok ، Human Examiners ، IELTS writing