Why we need to be careful with LLMs in medicine
2024

Why We Need to Be Careful with LLMs in Medicine

publication Evidence: high

Author Information

Author(s): Jean-Christophe Bélisle-Pipon

Primary Institution: Simon Fraser University

Hypothesis

The inaccuracies produced by large language models (LLMs) in healthcare should be reframed from 'hallucinations' to 'bullshit' to better understand their risks.

Conclusion

LLMs can generate plausible-sounding text but lack true understanding, which poses significant risks in medical applications.

Supporting Evidence

  • LLMs can produce text that appears coherent but lacks genuine understanding.
  • The term 'AI hallucination' may downplay the seriousness of inaccuracies in medical contexts.
  • Using LLMs without expert validation can lead to significant risks in patient care.
  • Transparency and critical assessment of AI outputs are essential for safe medical practice.

Takeaway

LLMs are like talking parrots that can say things that sound right but don't really know if they're true, which can be dangerous in medicine.

Potential Biases

There is a risk of over-reliance on LLMs, which may lead to misinformation in critical medical contexts.

Limitations

The article discusses the limitations of LLMs in understanding and verifying information, emphasizing the need for caution in their use in healthcare.

Digital Object Identifier (DOI)

10.3389/fmed.2024.1495582

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication