Article type
Year
Abstract
Background: Initial version of the Cochrane risk of bias (RoB) tool included one joint domain for “blinding of participants, personnel and outcome assessors”. In the 2011 version of the tool, this domain was split into two domains: blinding of participants and personnel (performance bias) and blinding of outcome assessors (detection bias). There are no studies that have compared whether the decision to split the domain was justified.
Objectives: We analyzed prevalence of usage of the joint blinding domain, and proportion of inadequate assessments made in the joint versus single RoB domains for blinding in Cochrane reviews.
Methods: This was a primary methodological study, in which we analyzed methodology of Cochrane reviews published in 2016/2017 in the Cochrane Database of Systematic Reviews (CDSR). From Cochrane reviews we extracted information about assessment of blinding from RoB tables, including judgment and comment. For data extraction we used a custom-designed software. We assessed prevalence of using joint blinding domain for performance and detection bias, and split domain where performance bias and detection bias were analyzed separately. We assessed whether judgments made by Cochrane authors were adequate by comparing judgments and comments with instructions from the Cochrane Handbook for Systematic Reviews of Interventions. We compared frequency of inadequate judgments in reviews with two separate domains for blinding, versus those with a single domain for blinding.
Results: We analyzed 728 Cochrane reviews, with 10523 trials included. In those Cochrane reviews, we found 6918 assessments for performance bias, 8656 for detection bias, and 3228 for the joint domain. Prevalence of adequate assessments was 73% for performance bias, 78% for detection bias and 59% for the joint domain. The lowest prevalence of adequate assessments was found when Cochrane authors made judgment of low risk – 47% in performance bias, 62% in detection bias and 31% in joint domain.
Conclusions: Decision to split single RoB domain for blinding of key individuals was justified, as Cochrane authors more frequently make adequate judgments with split domain for blinding. In Cochrane RoB 2.0 tool, blinding is assessed in three separate domains. We anticipate that this should result in even higher adequacy of judgments of blinding of key individuals in Cochrane RoB 2.0 tool, but this will need to be confirmed after its full implementation in Cochrane reviews.
Patient or healthcare consumer involvement: This was a research methodology study and it did not include patient or healthcare consumers.
Objectives: We analyzed prevalence of usage of the joint blinding domain, and proportion of inadequate assessments made in the joint versus single RoB domains for blinding in Cochrane reviews.
Methods: This was a primary methodological study, in which we analyzed methodology of Cochrane reviews published in 2016/2017 in the Cochrane Database of Systematic Reviews (CDSR). From Cochrane reviews we extracted information about assessment of blinding from RoB tables, including judgment and comment. For data extraction we used a custom-designed software. We assessed prevalence of using joint blinding domain for performance and detection bias, and split domain where performance bias and detection bias were analyzed separately. We assessed whether judgments made by Cochrane authors were adequate by comparing judgments and comments with instructions from the Cochrane Handbook for Systematic Reviews of Interventions. We compared frequency of inadequate judgments in reviews with two separate domains for blinding, versus those with a single domain for blinding.
Results: We analyzed 728 Cochrane reviews, with 10523 trials included. In those Cochrane reviews, we found 6918 assessments for performance bias, 8656 for detection bias, and 3228 for the joint domain. Prevalence of adequate assessments was 73% for performance bias, 78% for detection bias and 59% for the joint domain. The lowest prevalence of adequate assessments was found when Cochrane authors made judgment of low risk – 47% in performance bias, 62% in detection bias and 31% in joint domain.
Conclusions: Decision to split single RoB domain for blinding of key individuals was justified, as Cochrane authors more frequently make adequate judgments with split domain for blinding. In Cochrane RoB 2.0 tool, blinding is assessed in three separate domains. We anticipate that this should result in even higher adequacy of judgments of blinding of key individuals in Cochrane RoB 2.0 tool, but this will need to be confirmed after its full implementation in Cochrane reviews.
Patient or healthcare consumer involvement: This was a research methodology study and it did not include patient or healthcare consumers.