External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
2007

Validation of the AMSTAR Tool for Systematic Reviews

Sample size: 42 publication Evidence: moderate

Author Information

Author(s): Shea Beverley J., Bouter Lex M., Peterson Joan, Boers Maarten, Andersson Neil, Ortiz Zulma, Ramsay Tim, Bai Annie, Shukla Vijay K., Grimshaw Jeremy M.

Primary Institution: Community Information and Epidemiological Technologies (CIET), Ottawa, Ontario, Canada

Hypothesis

Can the AMSTAR tool effectively assess the methodological quality of systematic reviews?

Conclusion

The AMSTAR tool demonstrated satisfactory inter-observer agreement, reliability, and construct validity in assessing the quality of systematic reviews.

Supporting Evidence

  • The overall AMSTAR scores ranged from 0 to 10 with a mean of 4.6.
  • Inter-observer agreement for individual items ranged from moderate to almost perfect.
  • Nine items scored a kappa of >0.75, indicating high reliability.
  • The reliability of the total AMSTAR score was excellent with a kappa of 0.84.

Takeaway

The AMSTAR tool helps check if systematic reviews are done well, and it works pretty well according to this study.

Methodology

The AMSTAR tool was applied to 42 systematic reviews on gastro-esophageal reflux disease and other related conditions by multiple assessors.

Potential Biases

Potential bias due to missing data on conflicts of interest in the systematic reviews assessed.

Limitations

The study was limited to a small sample of systematic reviews in a specific clinical area and a small number of assessors.

Participant Demographics

The mean age of reviewers was 40.57 years, with 57% identified as methodology experts.

Statistical Information

P-Value

p<0.0002

Confidence Interval

95% CI: 3.7 to 5.6

Statistical Significance

p<0.05

Digital Object Identifier (DOI)

10.1371/journal.pone.0001350

Want to read the original?

Access the complete publication on the publisher's website

View Original Publication