Article type
Year
Abstract
Background: Risk of bias (RoB) of trials may affect the conclusions of systematic reviews. Since 2008, Cochrane reviews are expected to include a detailed assessment of bias for included trials using the RoB tool. It is not clear how stringently this is followed, or how much potential bias is present across Cochrane reviews.
Objectives: To assess the RoB among included trials in Cochrane reviews.
Methods: We searched the Cochrane Database of Systematic Reviews (CDSR) for all active, completed reviews. Using a customized automated algorithm (Visual Basic for Applications) we extracted RoB assessments of included trials. An 'unclear’ risk was imputed for missing assessments except where RoB was not assessed at all.
Results: In April 2011, the CDSR contained 4,594 reviews. We excluded withdrawn reviews (n = 204) and overviews of reviews (n = 3). Further, 3,013 (66%) reviews were excluded for lack of data [RoB assessment not performed (n = 2,599); no trials included (n = 411); other quality assessment tool used (n = 3)]. In total, 1,374 reviews encompassing 17,220 trials were included in our analyses. Of these, 39% of trials had RoB judgments reported for all domains; remainder had variable missing assessments. Overall 3%, 53%, and 44% of trials were considered to be 'low’, 'unclear’, and 'high’ risk, respectively. The rationale for the judgments was provided for 88% of the assessments. The number of trials classified as 'high’ has significantly decreased over time, while trials classified as 'low’ or 'unclear’ risk have gradually increased (p<0.0001). The domain most often assessed as 'low’ risk was 'incomplete outcome data’ 'blinding’ was most often assessed as 'high’ risk.
Conclusions: RoB assessments for most trials included in Cochrane reviews are 'unclear’, likely due to poor reporting at the trial level. Further, 97% of trials were assessed as 'unclear’ or 'high’ risk which raises questions regarding the sensitivity/ specificity of the RoB tool.
Objectives: To assess the RoB among included trials in Cochrane reviews.
Methods: We searched the Cochrane Database of Systematic Reviews (CDSR) for all active, completed reviews. Using a customized automated algorithm (Visual Basic for Applications) we extracted RoB assessments of included trials. An 'unclear’ risk was imputed for missing assessments except where RoB was not assessed at all.
Results: In April 2011, the CDSR contained 4,594 reviews. We excluded withdrawn reviews (n = 204) and overviews of reviews (n = 3). Further, 3,013 (66%) reviews were excluded for lack of data [RoB assessment not performed (n = 2,599); no trials included (n = 411); other quality assessment tool used (n = 3)]. In total, 1,374 reviews encompassing 17,220 trials were included in our analyses. Of these, 39% of trials had RoB judgments reported for all domains; remainder had variable missing assessments. Overall 3%, 53%, and 44% of trials were considered to be 'low’, 'unclear’, and 'high’ risk, respectively. The rationale for the judgments was provided for 88% of the assessments. The number of trials classified as 'high’ has significantly decreased over time, while trials classified as 'low’ or 'unclear’ risk have gradually increased (p<0.0001). The domain most often assessed as 'low’ risk was 'incomplete outcome data’ 'blinding’ was most often assessed as 'high’ risk.
Conclusions: RoB assessments for most trials included in Cochrane reviews are 'unclear’, likely due to poor reporting at the trial level. Further, 97% of trials were assessed as 'unclear’ or 'high’ risk which raises questions regarding the sensitivity/ specificity of the RoB tool.