Beyond Holistic Scoring: Psychometric Validation of A Multidimensional Analytic Rubric For Arabic Writing Assessment

Holistic scoring still dominates Arabic writing assessment, and the rubrics behind it are seldom validated with the psychometric rigor needed to trust the scores they produce. This study aimed to develop and validate a multidimensional analytic rubric for Arabic writing competence, providing empirical evidence of its reliability and construct validity. A quantitative instrument validation design was employed involving ۳۵۰ undergraduate students of Arabic Language Education from universities across three regions in Indonesia, selected through purposive sampling. Participants completed a standardized Arabic writing task scored using a ۲۱-indicator analytic rubric structured around four dimensions: text organization, cohesion and coherence, linguistic accuracy, and lexical-stylistic control. The validation process integrated Classical Test Theory (CTT), Confirmatory Factor Analysis (CFA), and Item Response Theory (IRT) via the Graded Response Model (GRM). CTT analysis yielded acceptable item-total correlations (r = ۰.۴۲۲-۰.۷۴۰) and high internal consistency (Cronbach’s α = ۰.۹۲). CFA supported a four-factor structure with excellent model fit (CFI = ۰.۹۸۹, TLI = ۰.۹۸۷, RMSEA = ۰.۰۲۱, SRMR = ۰.۰۲۸) and adequate composite reliability (CR = ۰.۸۰۱-۰.۸۴۸), though convergent validity was only partially established. GRM analysis demonstrated moderate to very high item discrimination (a = ۰.۹۸۹-۲.۵۱۴) with well-ordered category thresholds. By combining classical and modern psychometric techniques, this culturally grounded rubric achieves optimal measurement precision within the low-to-moderate proficiency range and fills a methodological gap that few studies on Arabic language education in Indonesia have addressed with comparable rigor.

کلیدواژه ها:

Arabic writing assessment ، Analytic rubric ، Classical Test Theory ، Item Response Theory ، Psychometric Evaluation