Class Prediction for High-Dimensional Class-Imbalanced Data
Author Information
Author(s): Blagus Rok, Lusa Lara
Primary Institution: Institute for Biostatistics and Medical Informatics, University of Ljubljana
Hypothesis
Does high-dimensionality pose additional challenges when dealing with class-imbalanced prediction?
Conclusion
Class imbalance exacerbates the challenges of classification in high-dimensional data, and matching class prevalence in training and test sets does not ensure good classifier performance.
Supporting Evidence
- Classifiers are sensitive to class imbalance, often misclassifying minority class samples.
- Variable selection can introduce bias towards the majority class.
- Down-sizing and asymmetric bagging can improve classifier performance under class imbalance.
Takeaway
When trying to predict classes from lots of data, if one class has way more examples than the other, it can mess up the predictions. We need to be careful about how we train our models.
Methodology
The study evaluated six types of classifiers on simulated and real breast cancer gene-expression data, focusing on their performance under class imbalance.
Potential Biases
The classifiers showed a tendency to favor the majority class, especially when variable selection was involved.
Limitations
The study primarily focused on specific classifiers and did not explore all possible methods for handling class imbalance.
Participant Demographics
The study used simulated data and a publicly available breast cancer gene-expression dataset.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website