To Tweak or Not to Tweak. How Exploiting Flexibilities in Gene Set Analysis Leads to Overoptimism
2024

Overoptimism in Gene Set Analysis

Sample size: 69 publication 10 minutes Evidence: moderate

Author Information

Author(s): Milena Wünsch, Christina Sauer, Moritz Herrmann, Ludwig Christian Hinske, Anne‐Laure Boulesteix

Primary Institution: Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine LMU Munich

Hypothesis

How does exploiting flexibilities in gene set analysis lead to overoptimism in research findings?

Conclusion

The study highlights that overoptimism in gene set analysis can lead to nonreplicable results, particularly when researchers cherry-pick analysis strategies.

Supporting Evidence

  • The study indicates that overoptimism is particularly high for GSEA-based methods.
  • Cherry-picking analysis strategies can lead to inflated false discovery rates.
  • Researchers with little statistical experience are more likely to engage in cherry-picking.

Takeaway

Researchers sometimes change their analysis methods to get better results, which can make their findings look better than they really are.

Methodology

The study mimics a hypothetical researcher tweaking gene set analysis results across various methods and data sets to assess the potential for overoptimism.

Potential Biases

The study acknowledges that cherry-picking analysis strategies can lead to biased and nonreplicable results.

Limitations

The study may underestimate the potential for overoptimism as it focuses only on the analysis stage and does not account for uncertainties during data generation.

Participant Demographics

The study used gene expression data from 69 Nigerian individuals, with 29 males and 40 females.

Digital Object Identifier (DOI)

10.1002/bimj.70016

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication