Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review
2024

Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care

Sample size: 65 publication 10 minutes Evidence: moderate

Author Information

Author(s): Mavragani Amaryllis, Pinjar Mohammed Jaffer, Kavadella Argyro, Xia Jing, Sengupta Saurav, Zielinski Chris, Wang Leyao, Wan Zhiyu, Ni Congning, Song Qingyuan, Li Yang, Clayton Ellen, Malin Bradley, Yin Zhijun

Primary Institution: Vanderbilt University Medical Center

Hypothesis

This review aims to summarize applications of and concerns regarding conversational LLMs in health care and provide an agenda for future research in this field.

Conclusion

Conversational LLMs like ChatGPT show promise in summarizing health-related texts and answering general health questions, but they struggle with complex diagnoses and recommendations.

Supporting Evidence

  • ChatGPT was the most popular LLM in the reviewed papers, appearing in 92% of them.
  • 75% of the papers used LLMs for summarization or medical knowledge inquiry.
  • 89% of the papers expressed concerns about reliability or bias.

Takeaway

ChatGPT can help doctors and patients by summarizing information and answering questions, but it sometimes gets things wrong, especially with complicated health issues.

Methodology

The review followed PRISMA guidelines to screen and select peer-reviewed research articles related to health care applications and conversational LLMs published before September 1, 2023.

Potential Biases

Concerns about bias include disparities in access and outcomes based on financial, readability, and accessibility barriers.

Limitations

The review primarily included studies focused on ChatGPT, limiting the generalizability of findings to other LLMs, and many concerns were based on speculation rather than rigorous experiments.

Digital Object Identifier (DOI)

10.2196/22769

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication