Article type
Year
Abstract
Background: The Cochrane Collaboration released recommendations for the conduct of SRs in 2011. They recommend using the Cochrane Risk of Bias (ROB) tool for assessing risk of bias in randomized controlled trials (RCT) but lack specific guidance for non-randomized studies.
Objectives: (1) describe approaches/tools used to assess risk of bias in a set of non-Cochrane reviews; (2) examine changes in approach over time; and (3) compare approaches to Cochrane recommendations.
Methods: We examined 50 reviews and 35 protocols produced through the US Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program (2005–2012). All reviews/protocols are accessible online. We extracted information on methods for risk of bias assessments, and compared methods over time.
Results: The most common approach was using individual components (26/50 reviews). Various sources were cited as contributing to the components selected: US Preventive Services Task Force, CONSORT, National Health Services Center for Reviews and Dissemination, the AHRQ/EPC methods guide (for trials); Newcastle-Ottawa Scale (NOS) and a publication by Deeks et al. (for observational studies). Many reviews (30/50) also provided summary assessments for each study most often in the form of good, fair, or poor based on guidance in the AHRQ/EPC methods guide. In some cases these summary assessments were design-specific while in other cases they considered all designs. The Jadad scale was used in 7/50 reviews, and 1/35 protocols. Cochrane ROB tool was used in 3/35 reviews, and cited in 9/35 protocols. The NOS was commonly cited (8/50 reviews, 7/35 protocols). In some reviews, different approaches were used to assess the same studies depending on questions of efficacy vs. safety.
Conclusions: Using individual components is the most common approach to assessing risk of bias in non-Cochrane reviews. Use of the Cochrane ROB tool is increasing for trials, while the NOS is common for observational studies.
Objectives: (1) describe approaches/tools used to assess risk of bias in a set of non-Cochrane reviews; (2) examine changes in approach over time; and (3) compare approaches to Cochrane recommendations.
Methods: We examined 50 reviews and 35 protocols produced through the US Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program (2005–2012). All reviews/protocols are accessible online. We extracted information on methods for risk of bias assessments, and compared methods over time.
Results: The most common approach was using individual components (26/50 reviews). Various sources were cited as contributing to the components selected: US Preventive Services Task Force, CONSORT, National Health Services Center for Reviews and Dissemination, the AHRQ/EPC methods guide (for trials); Newcastle-Ottawa Scale (NOS) and a publication by Deeks et al. (for observational studies). Many reviews (30/50) also provided summary assessments for each study most often in the form of good, fair, or poor based on guidance in the AHRQ/EPC methods guide. In some cases these summary assessments were design-specific while in other cases they considered all designs. The Jadad scale was used in 7/50 reviews, and 1/35 protocols. Cochrane ROB tool was used in 3/35 reviews, and cited in 9/35 protocols. The NOS was commonly cited (8/50 reviews, 7/35 protocols). In some reviews, different approaches were used to assess the same studies depending on questions of efficacy vs. safety.
Conclusions: Using individual components is the most common approach to assessing risk of bias in non-Cochrane reviews. Use of the Cochrane ROB tool is increasing for trials, while the NOS is common for observational studies.