Evaluation of the Claude AI Assistant's Performance on the Iranian Master's Entrance Exam in Medical Physics

Aim: This study aimed to assess the performance of the Claude AI assistant[۲] on a multiple choice exam covering key topics in medical physics and determine areas needing improvement. Methods: Claude was provided a ۱۶۰ question multiple choice exam from the Iranian Master's Entrance Exam in Medical Physics directly in PDF form[۱] without using any OCR tools. Claude provided its best reasoned answers, which were compared to the answer key to calculate percent correct overall and by topic. Results: Overall Claude achieved ۶۱% accuracy compared to the answer key. Performance was strongest in Physiology and Anatomy (۶۷% correct), radiation physics, general physics, and math (۶۰% each), and general English (۶۸%). Weaker areas were nuclear/atomic physics (۵۵% correct), radiobiology (۵۸%), biology (۶۰%), and physiology/anatomy (۶۷%). Conclusion: The Claude AI assistant demonstrated a foundational command of key physics topics, with room for improvement in specialized medical applications. Additional training focused on nuclear physics, radiobiology, and biological sciences would further enhance Claude's performance on medical physics exams and tasks requiring cross-disciplinary knowledge. However, Claude shows promise in integrating physics and medical concepts.

کلیدواژه ها:

artificial intelligence ، medical physics ، exam performance ، physics knowledge ، cross-disciplinary learning