Using a Large Language Model to Classify Dementia Caregivers’ Health Information Wants
Author Information
Author(s): Li Zhuochun, Luo Zhimeng, Zou Ning, He Daqing, Xie Bo, Hilsabeck Robin, Aguirre Alyssa
Primary Institution: University of Pittsburgh
Hypothesis
Can GPT-3.5 Turbo effectively classify health information wants expressed by dementia caregivers in social media posts?
Conclusion
GPT-3.5 Turbo showed limited accuracy in classifying health information wants from dementia caregivers' social media posts.
Supporting Evidence
- 60 social media posts were classified based on a health information wants framework.
- 23% of posts were accurately classified to daily care.
- 71% of the accurately classified posts were categorized into mood and behavior management or safety.
- 70% of the remaining posts were accurately classified into third-level subcategories.
Takeaway
The study tested a computer program to see if it could understand what caregivers of people with dementia need help with, but it didn't do very well.
Methodology
Evaluated GPT-3.5 Turbo's performance in classifying social media posts based on a health information wants framework.
Limitations
The performance of GPT-3.5 Turbo was disappointing, and the study discusses possible reasons for this.
Participant Demographics
Caregivers of persons living with dementia.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website