Article type
Year
Abstract
Background: Cochrane’s risk of bias (RoB) tool is used for assessment of randomized controlled trials (RCTs). The 2011 version of the tool had seven domains, including blinding of outcome assessors (detection bias). This domain, with modifications, is used in the new RoB 2.0 tool as well. Our research group has previously shown that Cochrane authors do not assess other RoB domains adequately in Cochrane reviews, but there were no prior studies about adequacy of judgments for detection bias in Cochrane reviews.
Objective: The aim of this study was to analyze whether judgments about the risk of bias associated with blinding of outcome assessors in Cochrane reviews of RCTs were adequate, i.e. in line with recommendations from the Cochrane Handbook.
Methods: This was a primary methodological study, in which we analyzed methodology of Cochrane reviews published in 2016/2017 in the Cochrane Database of Systematic Reviews (CDSR). Judgments and supporting comments for detection bias were extracted from RoB tables reported in Cochrane reviews. For data extraction we used customized software. We categorized comments, and then compared judgment and supporting comment with instructions from the Cochrane Handbook for Systematic Reviews of Interventions.
Results: We analyzed 8656 judgments for detection bias from 7626 trials included in 575 Cochrane reviews. In those 7626 trials there were 8656 domains (judgments) for detection bias, because in some Cochrane reviews this domain was split, i.e. had multiple assessments for various types of outcomes. Overall, 1909 judgments (22%) were not in line with the Cochrane Handbook. The highest prevalence of inadequate judgments was found for trials judged with low risk of detection bias (1287 of 3374; 38%), followed by those judged with high risk (239 of 1679; 14%) and those judged with unclear risk (383 of 3603; 11%).
In 9% of trials authors split the detection bias domain according to outcomes. Here, prevalence of inadequate judgments was 19%.
Conclusions: We found that Cochrane reviews frequently had inadequate judgments for risk of detection bias. It would be worthwhile to explore interventions that would help ensure adherence to methodological guidance among systematic review authors. Risk of bias judgments are incorporated into systematic review conclusions, and it is in the interest of the entire medical community to have trustworthy evidence.
Patient or healthcare consumer involvement: This was a research methodology study and it did not include patient or healthcare consumers.
Objective: The aim of this study was to analyze whether judgments about the risk of bias associated with blinding of outcome assessors in Cochrane reviews of RCTs were adequate, i.e. in line with recommendations from the Cochrane Handbook.
Methods: This was a primary methodological study, in which we analyzed methodology of Cochrane reviews published in 2016/2017 in the Cochrane Database of Systematic Reviews (CDSR). Judgments and supporting comments for detection bias were extracted from RoB tables reported in Cochrane reviews. For data extraction we used customized software. We categorized comments, and then compared judgment and supporting comment with instructions from the Cochrane Handbook for Systematic Reviews of Interventions.
Results: We analyzed 8656 judgments for detection bias from 7626 trials included in 575 Cochrane reviews. In those 7626 trials there were 8656 domains (judgments) for detection bias, because in some Cochrane reviews this domain was split, i.e. had multiple assessments for various types of outcomes. Overall, 1909 judgments (22%) were not in line with the Cochrane Handbook. The highest prevalence of inadequate judgments was found for trials judged with low risk of detection bias (1287 of 3374; 38%), followed by those judged with high risk (239 of 1679; 14%) and those judged with unclear risk (383 of 3603; 11%).
In 9% of trials authors split the detection bias domain according to outcomes. Here, prevalence of inadequate judgments was 19%.
Conclusions: We found that Cochrane reviews frequently had inadequate judgments for risk of detection bias. It would be worthwhile to explore interventions that would help ensure adherence to methodological guidance among systematic review authors. Risk of bias judgments are incorporated into systematic review conclusions, and it is in the interest of the entire medical community to have trustworthy evidence.
Patient or healthcare consumer involvement: This was a research methodology study and it did not include patient or healthcare consumers.