Article type
Year
Abstract
Background: Empirical evidence suggests that certain aspects of trial design may lead to biased intervention effect estimates.
Objectives: To examine the influence of 'Risk of bias' judgements from Cochrane Reviews for sequence generation, allocation concealment, blinding and incomplete data on intervention effect estimates in a large collection of meta-analyses (MAs).
Methods: We selected MAs with dichotomous outcomes and more than four included trials from intervention reviews with fully completed 'Risk of bias' tool, published in issue 4, 2011 of the Cochrane Library. We classified outcome measures as mortality, other objective or subjective, and estimated the effect of 'Risk of bias' domain judgements on average bias (ratios of odds ratios (ROR) with 95% credible intervals (CrI)) using Bayesian hierarchical models.
Results: Among 2815 trials in 256 meta-analyses, intervention effect estimates were on average exaggerated in trials with high or unclear risk of bias (versus low) for random sequence generation (ROR 0.91, 95% CrI 0.86 to 0.98), for allocation concealment (ROR 0.92, 95% CrI 0.86 to 0.98) and for blinding (ROR 0.87, 95% CrI 0.80 to 0.93). Unlike our previous study, we did not observe consistently different bias or between-trial heterogeneity in bias in MAs with subjective outcomes compared to mortality. Results from analyses of the influences of incomplete data were inconclusive.
Limitations: Possible inconsistency in criteria for 'Risk of bias' judgments applied by individual reviewers is a likely limitation of routinely collected bias assessments.
Conclusions: Inadequate randomization or lack of blinding may lead to exaggeration of intervention effect estimates in trials, but it is unclear if this effect differs by outcome type.
Objectives: To examine the influence of 'Risk of bias' judgements from Cochrane Reviews for sequence generation, allocation concealment, blinding and incomplete data on intervention effect estimates in a large collection of meta-analyses (MAs).
Methods: We selected MAs with dichotomous outcomes and more than four included trials from intervention reviews with fully completed 'Risk of bias' tool, published in issue 4, 2011 of the Cochrane Library. We classified outcome measures as mortality, other objective or subjective, and estimated the effect of 'Risk of bias' domain judgements on average bias (ratios of odds ratios (ROR) with 95% credible intervals (CrI)) using Bayesian hierarchical models.
Results: Among 2815 trials in 256 meta-analyses, intervention effect estimates were on average exaggerated in trials with high or unclear risk of bias (versus low) for random sequence generation (ROR 0.91, 95% CrI 0.86 to 0.98), for allocation concealment (ROR 0.92, 95% CrI 0.86 to 0.98) and for blinding (ROR 0.87, 95% CrI 0.80 to 0.93). Unlike our previous study, we did not observe consistently different bias or between-trial heterogeneity in bias in MAs with subjective outcomes compared to mortality. Results from analyses of the influences of incomplete data were inconclusive.
Limitations: Possible inconsistency in criteria for 'Risk of bias' judgments applied by individual reviewers is a likely limitation of routinely collected bias assessments.
Conclusions: Inadequate randomization or lack of blinding may lead to exaggeration of intervention effect estimates in trials, but it is unclear if this effect differs by outcome type.