Using AI to Detect Cognitive Decline in Older Adults
Author Information
Author(s): Du Xinsong, Novoa-Laurentiev John, Plasek Joseph M., Chuang Ya-Wen, Wang Liqin, Marshall Gad A., Mueller Stephanie K., Chang Frank, Datta Surabhi, Paek Hunki, Lin Bin, Wei Qiang, Wang Xiaoyan, Wang Jingqi, Ding Hao, Manion Frank J., Du Jingcheng, Bates David W., Zhou Li
Primary Institution: Brigham and Women's Hospital
Hypothesis
Can large language models effectively identify signs of cognitive decline in electronic health records?
Conclusion
The study found that an ensemble model combining large language models and traditional machine learning methods significantly improved the detection of cognitive decline in clinical notes.
Supporting Evidence
- The ensemble model outperformed individual models in all evaluation metrics.
- GPT-4 showed superior accuracy compared to Llama 2 but did not surpass traditional models.
- Error analysis revealed diverse error profiles among the models.
Takeaway
Researchers used AI to look for signs of memory problems in doctors' notes about older patients, and found that combining different types of AI made it better at spotting these issues.
Methodology
The study analyzed clinical notes from patients diagnosed with mild cognitive impairment, using large language models and traditional machine learning models to compare their performance.
Potential Biases
The study's reliance on electronic health records may introduce biases related to data quality and completeness.
Limitations
The models may not represent the latest advancements in AI, and the data used were not patient-based, which could affect the accuracy of distinguishing between reversible and progressive cognitive decline.
Participant Demographics
Patients aged 50 and older, with a mean age of 76 years; 53.1% were women.
Statistical Information
P-Value
<0.01
Confidence Interval
[95% CI: 81.9%–96.8%]
Statistical Significance
p<0.01
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website