Understanding the Matthews Correlation Coefficient
Author Information
Author(s): Itaya Yuki, Tamura Jun, Hayashi Kenichi, Yamamoto Kouji
Primary Institution: Keio University
Hypothesis
How can we improve the statistical inference of the Matthews correlation coefficient (MCC) for better evaluation of classifier performance?
Conclusion
The study introduces methods for constructing asymptotic confidence intervals for the Matthews correlation coefficient, enhancing its reliability in evaluating classifier performance.
Supporting Evidence
- The Matthews correlation coefficient is a reliable metric for evaluating classifier performance, especially in imbalanced datasets.
- Simulations showed that the Fisher's z method outperformed the Simple method for deriving confidence intervals.
- Real data analysis demonstrated the practical utility of the proposed methods in comparing binary classifiers.
Takeaway
This study helps us understand how to better measure the performance of classifiers using a special score called the Matthews correlation coefficient, which is really good at handling tricky situations where some classes are much bigger than others.
Methodology
The paper evaluates methods for constructing asymptotic confidence intervals for the MCC using simulations and real data analysis.
Potential Biases
Potential biases in the MCC may arise from imbalanced datasets and reliance on specific metrics.
Limitations
The study primarily focuses on binary classification and does not address multi-class scenarios.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website