Estimating Power in Log-Linear Models for Ordinal Agreement
Author Information
Author(s): Fabien Valet, Mary Jean-Yves
Primary Institution: Institut Curie, Ecole des Mines de Paris, INSERM U900, Paris, FRANCE
Hypothesis
How many objects need to be classified by two observers to detect a scale structure defect?
Conclusion
The study highlights the importance of sample size and marginal homogeneity for detecting scale structure defects in ordinal rating scales.
Supporting Evidence
- For sample sizes of N = 50, the probabilities of detecting heterogeneities are lower than .80.
- Power estimates were greater than 80% for tested odds ratios greater than or equal to 12 for larger sample sizes.
- Marginal heterogeneities within raters led to a decrease in power estimates.
Takeaway
This study helps us understand how many things we need to look at to see if two people agree on their ratings, especially when using a scale with different levels.
Methodology
Simulations were used to estimate the power of non-uniform association models to detect heterogeneities across distinguishabilities between adjacent categories.
Limitations
The assumption of marginal homogeneity may not hold in all cases, which could affect the applicability of the results.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website