Examiner Leniency and Stringency in Clinical Examinations
Author Information
Author(s): IC McManus, M Thompson, J Mollon
Primary Institution: University College London
Hypothesis
Examiners differ in their leniency or stringency, affecting candidate outcomes in the MRCP(UK) clinical examination.
Conclusion
Examiners do differ in their leniency or stringency, and the effect can be estimated using Rasch modelling.
Supporting Evidence
- 87% of variance in marks was due to candidate differences.
- 12% of variance was due to differences between examiners.
- 2.6% of candidates who failed based on raw marks would have passed using adjusted marks.
Takeaway
Some examiners are stricter than others, which can change whether a student passes or fails an exam.
Methodology
Multi-facet Rasch modelling was used to analyze data from the first nine diets of PACES, involving 10,145 candidates assessed by 1,259 examiners.
Potential Biases
Potential biases due to demographic factors of examiners and candidates.
Limitations
The analysis may not account for interaction effects between examiners and candidates.
Participant Demographics
67.4% male and 32.6% female candidates; 44.2% were graduates of UK medical schools.
Statistical Information
P-Value
0.076 for sex difference; 0.003 for number of candidates examined.
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website