Evaluating GPT4 for Diagnosing Orofacial Pain
Author Information
Author(s): Vueghs Charlotte, Shakeri Hamid, Renton Tara, Van der Cruyssen Frederic, Dipalma Gianna
Primary Institution: University Hospitals Leuven
Hypothesis
Can GPT4 accurately diagnose orofacial pain compared to human clinicians?
Conclusion
GPT4 shows promise as a clinical decision support system for orofacial pain, improving diagnostic accuracy but not yet surpassing expert clinicians.
Supporting Evidence
- GPT4 achieved the highest accuracy level in 38% of cases.
- The model provided accurate differential diagnoses in 80% of cases.
- Interrater reliability between GPT4 and human evaluators was low (κ = 0.219).
- GPT4 correctly diagnosed 81% of trigeminal neuralgia cases at the highest accuracy level.
Takeaway
GPT4 is like a smart helper for doctors trying to figure out why someone has face pain, but it still needs to learn more to be as good as the best doctors.
Methodology
The study used 100 anonymized patient cases to evaluate GPT4's diagnostic accuracy against human evaluators.
Potential Biases
Potential biases in training data could affect diagnostic accuracy, especially for marginalized groups.
Limitations
The dataset may not cover all orofacial pain presentations, and GPT4's reliance on text inputs may miss important diagnostic clues.
Participant Demographics
44 males and 56 females, average age 48 years, ranging from 16 to 86 years.
Statistical Information
P-Value
p<0.001
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website