Comparing Two Standard-Setting Methods in Medical Education
Author Information
Author(s): George Sanju, Haque M Sayeed, Oyebode Femi
Primary Institution: Queen Elizabeth Psychiatric Hospital, University of Birmingham
Hypothesis
Do different standard-setting methods result in different standards?
Conclusion
The study found significant differences in the outcomes of the norm-reference and modified Angoff methods, with the Angoff method yielding a higher pass rate.
Supporting Evidence
- The pass rate with the norm-reference method was 85% and that by the Angoff method was 100%.
- The percentage agreement between the Angoff method and norm-reference was 78%.
- The modified Angoff method had an inter-rater reliability of 0.81 – 0.82.
- The test-retest reliability of the modified Angoff method was 0.59–0.74.
Takeaway
This study looked at two ways to decide if medical students passed a test and found that one way said all students passed while the other said some failed.
Methodology
The study compared the norm-reference method and the modified Angoff method using scores from 78 medical students on a multiple-choice exam.
Potential Biases
Potential bias due to the small number of judges and the arbitrary nature of the cut-off score.
Limitations
The small number of raters and the arbitrary choice of mean minus 1 SD as the cut-off score.
Participant Demographics
Fourth-year medical students from Birmingham medical school.
Statistical Information
P-Value
<0.001
Confidence Interval
95% CI 69% – 87%
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website