Article type
Year
Abstract
Background: AMSTAR (A Measurement Tool to Assess Systematic Reviews) is widely used in the critical appraisal of systematic reviews (SRs). The instrument was validated using SR of randomized controlled trials (RCTs). Surveys have shown that over 60% of SRs of treatment effects include non-randomized studies (NRS).
Objectives and methods: To develop a version of AMSTAR capable of appraising SRs that include NRS. A literature review found no critical appraisal instrument that incorporated suitable measures of risk of bias (RoB) of the component studies. An expert group reviewed the results and suggested a single instrument with separate paths to incorporate RoB assessment of either RCTs or NRS. The opportunity was taken to update the AMSTAR instrument in a number of other domains.
Results: The modified AMSTAR instrument now has 15 items. The review process is guided more clearly by the PICOT framework (i.e. population, intervention, comparison, outcome, and timeframe), makes a more detailed assessment of literature searching methods, gives guidance on whether authors have made adequate assessment of RoB of included studies and whether limitations were acknowledged in making conclusions from the results of the review. The domains of bias (modified from those in the ACROBAT-NRSI tool) are: freedom from residual or unmeasured confounding; freedom from selection bias; freedom from bias in measuring exposures and outcomes; freedom from selective outcome/analysis reporting. The revised instrument is undergoing pilot testing.
Conclusions: assessing the quality of SRs that include NRS is very challenging. The modified AMSTAR is a first step towards a more accurate appraisal of this important type of study.
Objectives and methods: To develop a version of AMSTAR capable of appraising SRs that include NRS. A literature review found no critical appraisal instrument that incorporated suitable measures of risk of bias (RoB) of the component studies. An expert group reviewed the results and suggested a single instrument with separate paths to incorporate RoB assessment of either RCTs or NRS. The opportunity was taken to update the AMSTAR instrument in a number of other domains.
Results: The modified AMSTAR instrument now has 15 items. The review process is guided more clearly by the PICOT framework (i.e. population, intervention, comparison, outcome, and timeframe), makes a more detailed assessment of literature searching methods, gives guidance on whether authors have made adequate assessment of RoB of included studies and whether limitations were acknowledged in making conclusions from the results of the review. The domains of bias (modified from those in the ACROBAT-NRSI tool) are: freedom from residual or unmeasured confounding; freedom from selection bias; freedom from bias in measuring exposures and outcomes; freedom from selective outcome/analysis reporting. The revised instrument is undergoing pilot testing.
Conclusions: assessing the quality of SRs that include NRS is very challenging. The modified AMSTAR is a first step towards a more accurate appraisal of this important type of study.