Examiner Score Comparison and Adjustment in OSCEs
Author Information
Author(s): Peter Yeates, Gareth McCray
Primary Institution: Keele University
Hypothesis
The study aims to investigate the accuracy of adjusted scores produced by Video-based Examiner Score Comparison and Adjustment (VESCA) under various operational parameters.
Conclusion
Score adjustment through VESCA can significantly improve score accuracy in some scenarios while potentially worsening it in others.
Supporting Evidence
- Score adjustment improved accuracy by up to 71% when baseline differences were significant.
- 93% of students had more accurate scores in the 20% baseline-difference condition.
- Without baseline differences, score adjustment often worsened accuracy.
Takeaway
This study looks at how to make exam scores fairer by comparing how different teachers grade students using videos. Sometimes it helps, but sometimes it can make things worse.
Methodology
The study used statistical simulation to analyze how various factors like examiner participation and baseline differences in stringency affect score accuracy.
Potential Biases
Potential biases may arise from the assumptions made in the simulation regarding examiner stringency and student performance.
Limitations
The study's simulations were limited to specific parameters and may not account for all possible influences on OSCE scores.
Participant Demographics
Participants were Year 3 medical students from Keele University.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website