Error in statistical tests of error in statistical tests
2006
Errors in Statistical Tests
Sample size: 610
publication
Evidence: moderate
Author Information
Author(s): Jeng Monwhea
Primary Institution: Southern Illinois University Edwardsville
Hypothesis
Are the statistical tests used in published papers valid?
Conclusion
Statistical tests should be used correctly.
Supporting Evidence
- The original paper's test of equiprobability was based on invalid use of the Kolmogorov-Smirnov test.
- Correct statistical testing finds no statistically significant deviations from equiprobability.
- Statistical tests in papers need to be inspected more closely.
Takeaway
This study found that some statistical tests in papers were done incorrectly, which means we need to be careful when looking at numbers in research.
Methodology
We checked the statistical analysis behind the original paper's tests of equiprobability using χ2 tests and a modified Kolmogorov-Smirnov test.
Potential Biases
The original paper's conclusions were based on an invalid statistical test.
Limitations
The analysis does not change the finding of some incorrect p values in Nature.
Statistical Information
P-Value
p < 0.0005
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website