Himel Mondal - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Anup Kumar Dhanvijay - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Ayesha Juhi - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Amita Singh - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Mohammed Jaffer Pinjar - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Anita Kumari - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Swati Mittal - Department of Physiology, Kalyan Singh Government Medical College Bulandshahr, Uttar Pradesh, India
Amita Kumari - Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
Shaikat Mondal - Department of Physiology, Raiganj Government Medical College and Hospital, West Bengal, India

Background: There has been increasing interest in exploring the capabilities of artificial intelligence (AI) in various fields, including education. Medical education is an area where AI can potentially have a significant impact, especially in helping students answer their customized questions. In this study, we aimed to investigate the capability of ChatGPT, a conversational AI model in generating answers to medical physiology exam questions in an Indian medical school.Methods: This cross-sectional study was conducted in March ۲۰۲۳ in an Indian Medical School, Deoghar, Jharkhand, India. The first mid-semester physiology examination was taken as the reference examination. There were two long essays, five short essay questions (total mark ۴۰), and ۲۰ multiple-choice questions (MCQ) (total mark ۱۰). We generated the response from ChatGPT (in March ۱۳ version) for both essay and MCQ questions. The essay-type answer sheet was evaluated by five faculties, and the average was taken as the final score. The score of ۱۲۵ students (all first-year medical students) in the examination was obtained from the departmental registery. The median score of the ۱۲۵ students was compared with the score of ChatGPT using Mann-Whitney U test.Results: The median score of ۱۲۵ students in essay-type questions was ۲۰.۵ (Q۱-Q۳: ۱۸-۲۳.۵) which corresponds to a median percentage of ۵۱.۲۵% (Q۱-Q۳: ۴۵-۵۸.۷۵) (P=۰.۱۴۷). The answer generated by ChatGPT scored ۲۱.۵ (Q۱-Q۳: ۲۱.۵-۲۲), which corresponds to ۵۳.۷۵% (Q۱-Q۳: ۵۳.۷۵-۵۵) (P=۰.۱۲۵). Hence, ChatGPT scored like that of the students (P=۰.۴) in essay-type questions. In MCQ-type questions, ChatGPT answered ۱۹ correctly in ۲۰ questions (score=۹.۵), and this was higher than the median score of students (۶) (Q۱-Q۳: ۵-۶.۵) (P<۰.۰۰۰۱).Conclusion: ChatGPT has the potential to generate answers to medical physiology examination questions. It has a higher capability to solve MCQ questions than essay-type ones. Although ChatGPT was able to provide answers that had the quality to pass the examination, the capability of generating high-quality answers for educational purposes is yet to be achieved. Hence, its usage in medical education for teaching and learning purposes is yet to be explored.

Distance, education, Artificial Intelligence, ChatGPT, Physiology, examination, Students, Medical