Evaluating AI Information in Plastic Surgery
Author Information
Author(s): Muacevic Alexander, Adler John R, Keating Muireann, Bollard Stephanie M, Potter Shirley
Primary Institution: St James's Hospital, Dublin, IRL
Hypothesis
The study aims to evaluate the quality and readability of information given by ChatGPT-4 on key areas in plastic and reconstructive surgery.
Conclusion
ChatGPT-4 produced low-quality information on plastic and aesthetic surgery queries, but was rated well in clarity and accuracy by clinicians.
Supporting Evidence
- All answers were scored as poor or very poor according to the DISCERN tool.
- The mean DISCERN score for all questions was 34.
- The mean FKRE index was 33.6, indicating low readability.
- Clinicians rated the mean clarity score as 7.38 and accuracy score as 7.4.
Takeaway
This study looked at how well ChatGPT answers questions about plastic surgery. It found that while the answers weren't very good, doctors thought they were clear and accurate.
Methodology
The study used Google Trends to identify popular plastic surgery topics, created questions, and evaluated ChatGPT-4 responses using the DISCERN tool and readability indices.
Potential Biases
ChatGPT may produce factually incorrect information and lacks proper citation, which can mislead patients.
Limitations
The study was limited by the number of participants and was conducted in a single center.
Participant Demographics
12 physicians, including 6 consultants and 6 trainees, working in a Plastic and Reconstructive Surgery unit.
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website