Using ChatGPT for Orthopaedic Exam Preparation
Author Information
Author(s): Dhruv Mendiratta, Isabel Herzog, Rohan Singh, Ashok Para, Tej Joshi, Michael Vosbikian, Neil Kaushal
Primary Institution: Department of Orthopaedic Surgery Rutgers New Jersey Medical School Newark New Jersey USA
Hypothesis
This study assesses ChatGPT's performance on the Orthopaedic In‐Training Exam (OITE) for use as a potential study resource for residents.
Conclusion
ChatGPT demonstrates logical, informational, and explicit fallacies which may lead to misinformation and hinder resident education.
Supporting Evidence
- ChatGPT achieved a success rate of 48.3% on the OITE.
- Logical reasoning was used in 67.6% of correct responses.
- Internal information was utilized in 68.1% of questions.
- External information was incorporated in 68.1% of questions.
- Informational fallacy was the most common error type.
Takeaway
ChatGPT can answer some questions for orthopaedic exams, but it often makes mistakes that could confuse students.
Methodology
ChatGPT was evaluated on 207 questions from the 2022 OITE, with responses categorized based on logical reasoning and fallacies.
Potential Biases
Researcher bias may have influenced findings, as ChatGPT could have used input from previous questions.
Limitations
The study used only one OITE practice exam, which may limit external validity.
Participant Demographics
Orthopaedic surgery residents preparing for the OITE.
Statistical Information
P-Value
p<0.001
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website