Various criteria in the evaluation of biomedical named entity recognition
2006

Evaluating Biomedical Named Entity Recognition

Sample size: 404 publication Evidence: moderate

Author Information

Author(s): Tsai Richard Tzong-Han, Wu Shih-Hung, Chou Wen-Chi, Lin Yu-Chun, He Ding, Hsiang Jieh, Sung Ting-Yi, Hsu Wen-Lian

Primary Institution: Institute of Information Science, Academia Sinica

Hypothesis

How do different matching criteria affect the performance evaluation of biomedical named entity recognition systems?

Conclusion

The study found that right match and left match can serve as effective alternatives to traditional matching criteria in biomedical named entity recognition.

Supporting Evidence

  • The study compared the performance of systems using different matching criteria.
  • Right match and left match were found to be effective alternatives to exact match.
  • The relaxed-match criterion improved the evaluation of Bio-NER systems.

Takeaway

This study looks at how to better identify names of genes and proteins in medical texts by using different ways to check if the names are correct.

Methodology

The study conducted four experiments comparing different matching criteria on datasets from JNLPBA and BioCreAtIvE.

Potential Biases

The reliance on specific matching criteria may introduce bias in evaluating system performance.

Limitations

The evaluation may not fully recreate the original class definitions used in BioCreAtIvE, leading to potential discrepancies in performance assessment.

Statistical Information

P-Value

0.05

Statistical Significance

p<0.05

Digital Object Identifier (DOI)

10.1186/1471-2105-7-92

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication