Article type
Year
Abstract
Background: In 2008 The Cochrane Collaboration introduced the Risk of Bias (RoB) tool to assess internal validity of randomized controlled trials (RCTs). Modifications to the tool were released in 2011 after user testing and feedback.
Objectives: To describe the results of applying the modified RoB tool.
Methods: Two researchers independently applied the tool to 204 RCTs. Disagreements were resolved through consensus. We included several modifications to the original tool. First, we assessed blinding separately for participants, investigators, and outcome assessors. Second, we assessed potential influence of study sponsorship separately from óther sources of bias'. We assessed agreement across blinding domains using kappa. We assessed correlation between different blinding domains, and between funding and óther’ domains using Kendall’s Tau.
Results: Risk of bias assessments varied among the three blinding domains. Of note was the higher frequency of low risk of bias for blinding of outcome assessors (51%) compared with blinding of participants (24%) and investigators (31%). The agreement for the blinding domains was fair (κ=0.32). The pair-wise correlation for the blinding domains was moderate (τ=0.39), participant versus investigator; (τ=0.38), participant versus outcome assessor; (τ=0.45), investigator versus outcome assessor). For other sources of bias, risk of bias was low in 118 (58%), high in 33 (16%), and unclear in 53 (26%). For sources of funding risk of bias was low in 77 (38%), high in 14 (7%), and unclear in 113 (55%). The correlation between 'other sources of bias' and 'source of funding’ was weak (τ=0.04). Researchers applying the RoB tool found it easier to assess blinding as three separate domains rather than a single item.
Conclusions: This study provides evidence that risk of bias due to blinding varies depending on the targeted individuals. Risk of bias due to 'other’ sources is different from inappropriate influence of study sponsor.
Objectives: To describe the results of applying the modified RoB tool.
Methods: Two researchers independently applied the tool to 204 RCTs. Disagreements were resolved through consensus. We included several modifications to the original tool. First, we assessed blinding separately for participants, investigators, and outcome assessors. Second, we assessed potential influence of study sponsorship separately from óther sources of bias'. We assessed agreement across blinding domains using kappa. We assessed correlation between different blinding domains, and between funding and óther’ domains using Kendall’s Tau.
Results: Risk of bias assessments varied among the three blinding domains. Of note was the higher frequency of low risk of bias for blinding of outcome assessors (51%) compared with blinding of participants (24%) and investigators (31%). The agreement for the blinding domains was fair (κ=0.32). The pair-wise correlation for the blinding domains was moderate (τ=0.39), participant versus investigator; (τ=0.38), participant versus outcome assessor; (τ=0.45), investigator versus outcome assessor). For other sources of bias, risk of bias was low in 118 (58%), high in 33 (16%), and unclear in 53 (26%). For sources of funding risk of bias was low in 77 (38%), high in 14 (7%), and unclear in 113 (55%). The correlation between 'other sources of bias' and 'source of funding’ was weak (τ=0.04). Researchers applying the RoB tool found it easier to assess blinding as three separate domains rather than a single item.
Conclusions: This study provides evidence that risk of bias due to blinding varies depending on the targeted individuals. Risk of bias due to 'other’ sources is different from inappropriate influence of study sponsor.