Estimation of minimal data sets sizes for machine learning predictions in digital mental health interventions
2024

Minimum Data Set Sizes for Machine Learning in Digital Mental Health

Sample size: 3654 publication Evidence: high

Author Information

Author(s): Kirsten Zantvoort, Barbara Nacke, Dennis Görlich, Silvan Hornstein, Corinna Jacobi, Burkhardt Funk

Primary Institution: Leuphana University, Lüneburg, Germany

Hypothesis

What are the minimal necessary data set sizes for machine learning predictions in digital mental health interventions?

Conclusion

The study proposes minimum dataset sizes of 500 to 1000 to improve predictive power and reduce overfitting in machine learning models for digital mental health interventions.

Supporting Evidence

  • Small datasets (N≤300) overestimate predictive power.
  • Overfitting was reduced with larger datasets (N≥500).
  • Performance did not converge until N=750-1500 for more sophisticated models.
  • Simple models were less likely to overfit but less competitive at larger sizes.
  • Complex features provided marginal benefits at larger dataset sizes.

Takeaway

To make good predictions about mental health interventions using computers, we need to collect enough data—at least 500 to 1000 pieces of information from users.

Methodology

The study analyzed learning curves for dropout predictions using data from 3654 users, evaluating prediction performance based on dataset size, feature groups, and algorithm choice.

Potential Biases

The sample may not represent the general population as it only includes self-referred female participants.

Limitations

The study is based on a single dataset and does not evaluate external generalizability; it also focuses only on self-referred female participants.

Participant Demographics

The participants were adult women without full-syndrome eating disorders, recruited from the general population.

Statistical Information

P-Value

0.005

Statistical Significance

p<0.05

Digital Object Identifier (DOI)

10.1038/s41746-024-01360-w

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication