Examining the Validity of AI-Based Speaking Evaluation: A Comparative Study of Automated and Human Ratings of Iranian EFL Learners' Oral Performance

Recent advancements in artificial intelligence (AI) and large language models (LLMs) have opened new possibilities for evaluating language speaking skills. As speech recognition and natural language processing technologies have matured, AI systems can now analyze pronunciation, fluency, lexical diversity, and coherence-dimensions traditionally assessed by human examiners. The current study investigates the extent to which AI-generated scores align with human ratings in the evaluation of Iranian EFL learners' oral performance on an IELTS-style speaking task. A total of ۸۰ Iranian learners participated in the study and performed standardized speaking tasks that were evaluated independently by both AI-based systems and experienced human raters using the IELTS Speaking Band Descriptors. Descriptive statistics, Pearson correlation, and paired-samples t-tests were conducted to examine consistency and scoring differences. The results revealed strong correlations between AI-derived and human ratings for pronunciation and fluency, moderate correlations for lexical resource, and relatively weaker alignment for coherence and cohesion. The findings suggest that AI-based speaking assessment systems hold promise for preliminary evaluation and diagnostic purposes in language testing, but they should complement rather than replace human raters, especially in high-stakes contexts where interpretive judgment is crucial.

کلیدواژه ها:

Artificial Intelligence ، Automated Speaking Evaluation ، ChatGPT ، Speech Assessment ، IELTS Speaking ، Oral Performance ، EFL Testing