Guide to Agreement and Reliability of Multi-Observer Ordinal Data
Author Information
Author(s): Vanbelle Sophie, Engelhart Christina Hernandez, Blix Ellen
Primary Institution: Maastricht University
Hypothesis
This paper aims to provide necessary information regarding the choice of meaningful measures and planning of agreement and reliability studies for ordinal outcomes.
Conclusion
The paper enhances existing guidelines to improve the quality of reliability and agreement studies of clinical tests and provides open-source software for researchers.
Supporting Evidence
- The paper addresses common issues in reliability and agreement studies for ordinal data.
- It provides a clear interpretation of agreement and reliability coefficients.
- Open-source software is made available to assist researchers with minimal programming skills.
Takeaway
This study helps researchers understand how to measure agreement and reliability when multiple observers are involved in assessing the same thing, like a medical test.
Methodology
The paper reviews statistical measures for agreement and reliability in ordinal scales and provides methods for sample size determination.
Limitations
The study does not provide analytical formulas for sample size calculations and relies on simulations.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website