The association of three bias domains with treatment effect estimates in randomised control trials: combined analysis of meta-epidemiological studies

Savovic J, Harris R, Brando Collaborators T

Background: Several meta-epidemiological studies have provided evidence that characteristics of randomised controlled trials (RCTs) are associated with intervention effect estimates. The new Cochrane Handbook for Systematic Reviews of Interventions includes a tool for assessing the risk of bias in RCT results. Objectives: To examine associations of adequacy of sequence generation, allocation concealment, and blinding with intervention effect estimates, and whether the magnitude of these associations varies with type of clinical area, intervention, comparison and outcome. Methods: Data from seven previous meta-epidemiological studies were combined. Duplicated and overlapping meta-analyses were removed. We used logistic regression and random-effects meta-analysis models to estimate ratios of odds ratios (ROR) comparing intervention effects in trials with and without different characteristics, allowing for between-meta-analysis heterogeneity in the effects of bias. Results: The ROR comparing inadequate/unclear with adequate random sequence generation was 0.87 (95% CI 0.81 to 0.94; between meta-analyses heterogeneity tau² = 0.043, P = 0.0006), based on 1038 trials from 119 meta-analyses. Average bias associated with inadequate/unclear sequence generation appeared less in trials where the outcome was all-cause mortality (ROR 0.94, 95% CI 0.84 to 1.05) than in trials with other outcomes (ROR 0.86, 95% CI 0.79 to 0.95). Consistent with recently reported results based on three of the studies combined here (BMJ 2008;336: 601–605), there was little evidence of between-meta-analysis heterogeneity in bias for trials with all-cause mortality outcomes, but clear heterogeneity (P < 0.001) for RCTs with other types of outcome. These patterns were similar for allocation concealment. There was little evidence that lack of blinding changed average intervention effect estimates, but clear between-meta-analysis heterogeneity in bias for RCTs with nonmortality outcomes. Analyses estimating the effect of each characteristic controlling for the others are in progress and will be presented. Conclusions: Bias associated with specific flaws in the conduct of RCTs varies between meta-analyses and is less in trials with all-cause mortality outcomes than in other outcomes. Assessments of the risk of bias in RCTs should account for these findings.