AI Curation in Biomedicine
Author Information
Author(s): Rodriguez-Esteban Raul, Iossifov Ivan, Rzhetsky Andrey
Primary Institution: Columbia University
Hypothesis
Using a larger number of human experts to evaluate any given sentence could allow for an artificial-intelligence curator to perform the classification job at least as accurately as an average individual human evaluator.
Conclusion
The study developed an AI program that closely approaches human experts in accurately distinguishing correctly extracted facts from incorrectly extracted ones.
Supporting Evidence
- The AI program achieved a ROC score close to 0.95, indicating high accuracy.
- The study used almost 100,000 independent evaluations to train the AI.
- The AI's performance was statistically indistinguishable from that of human evaluators.
Takeaway
Scientists created a computer program that can check facts in medical texts almost as well as humans can.
Methodology
The study used machine learning techniques to analyze a large training set of manually evaluated facts to develop an AI tool for fact evaluation.
Potential Biases
The training data may not represent all types of biological relationships, potentially affecting the AI's performance.
Limitations
The study's approach is biased toward relationships captured by the GeneWays system, which may not generalize to all biomedical texts.
Participant Demographics
The evaluations were performed by a team of human curators, including biology-savvy doctoral-level curators.
Statistical Information
P-Value
0.05
Statistical Significance
p<0.05
Digital Object Identifier (DOI)
Want to read the original?
Access the complete publication on the publisher's website