ROC and confusion analysis of structure comparison methods identify the main causes of divergence from manual protein classification
2006

Analyzing Protein Structure Comparison Methods

Sample size: 4676 publication 15 minutes Evidence: moderate

Author Information

Author(s): Sam Vichetra, Tai Chin-Hsien, Garnier Jean, Gibrat Jean-Francois, Lee Byungkook, Munson Peter J

Primary Institution: National Institutes of Health

Hypothesis

How do automated structure comparison methods diverge from manual protein classification?

Conclusion

The study reveals significant divergence between automated similarity measures and manual classifications, suggesting a continuous protein fold space.

Supporting Evidence

  • The ROC AUC for SHEBA was 0.93, indicating better performance than VAST.
  • At a 1% false positive rate, SHEBA achieved a true positive rate of 74.8%.
  • The study analyzed over 20 million pairs of protein domains.

Takeaway

This study looks at how well computer programs can tell if two proteins are similar based on their shapes, and finds that they often get it wrong.

Methodology

The study used ROC curves and confusion matrices to analyze the performance of two structure comparison methods, VAST and SHEBA, against SCOP classifications.

Potential Biases

Potential biases arise from the reliance on specific scoring methods that may not capture all relevant structural features.

Limitations

The methods may not account for all structural variations and ambiguities in protein fold definitions.

Statistical Information

Statistical Significance

p<0.05

Digital Object Identifier (DOI)

10.1186/1471-2105-7-206

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication