Article type
Year
Abstract
Background: A number of 'meta-epidemiological' studies have provided empirical evidence on sources of bias in randomized controlled trials. However, findings vary between these studies.
Objectives: To examine the associations of the adequacy of reported allocation concealment and blinding with treatment effect estimates in a data set combined from three previously published meta-epidemiological studies, and to examine whether the magnitude of these associations varies with the nature of the outcome variable.
Methods: Data were combined from three previously published meta-epidemiological studies (Schulz et al1, Kjaergard et al2, Egger et al3). Duplicated or overlapping meta-analyses were removed. We used logistic regression and random-effects meta-analysis to estimate ratios of odds ratios (ROR): an ROR<1 corresponds to more beneficial treatment effect estimates in lower quality trials.
Results: A total of 804 trials in 102 meta-analyses contributed to the estimated ROR comparing inadequate/unclear with adequate allocation concealment (ROR 0.83, 95% CI 0.74 to 0.93). Based on 746 trials in 76 meta-analyses, the ROR comparing inadequately with adequately blinded trials was 0.93 (95% CI 0.83 to 1.04). In meta-analyses with objective outcomes, there was a modest effect of allocation concealment (ROR 0.91, 95% CI 0.80 to 1.03) and no effect of blinding (1.01, 95% CI 0.92 to 1.10). In contrast, the effects of allocation concealment (0.69, 95% CI 0.59 to 0.82) and blinding (0.75, 95% CI 0.61 to 0.93) were substantial in meta-analyses with subjective outcomes. Meta-analyses assessing all-cause mortality showed little evidence of associations of either allocation concealment or blinding with treatment effect estimates.
Conclusions: Bias associated with aspects of trial quality may vary with the setting of the trial and in particular with the type of outcome variable.
References
1. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273:408-12.
2. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2001; 135:982-9.
3. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003; 7(1):1-68.
Objectives: To examine the associations of the adequacy of reported allocation concealment and blinding with treatment effect estimates in a data set combined from three previously published meta-epidemiological studies, and to examine whether the magnitude of these associations varies with the nature of the outcome variable.
Methods: Data were combined from three previously published meta-epidemiological studies (Schulz et al1, Kjaergard et al2, Egger et al3). Duplicated or overlapping meta-analyses were removed. We used logistic regression and random-effects meta-analysis to estimate ratios of odds ratios (ROR): an ROR<1 corresponds to more beneficial treatment effect estimates in lower quality trials.
Results: A total of 804 trials in 102 meta-analyses contributed to the estimated ROR comparing inadequate/unclear with adequate allocation concealment (ROR 0.83, 95% CI 0.74 to 0.93). Based on 746 trials in 76 meta-analyses, the ROR comparing inadequately with adequately blinded trials was 0.93 (95% CI 0.83 to 1.04). In meta-analyses with objective outcomes, there was a modest effect of allocation concealment (ROR 0.91, 95% CI 0.80 to 1.03) and no effect of blinding (1.01, 95% CI 0.92 to 1.10). In contrast, the effects of allocation concealment (0.69, 95% CI 0.59 to 0.82) and blinding (0.75, 95% CI 0.61 to 0.93) were substantial in meta-analyses with subjective outcomes. Meta-analyses assessing all-cause mortality showed little evidence of associations of either allocation concealment or blinding with treatment effect estimates.
Conclusions: Bias associated with aspects of trial quality may vary with the setting of the trial and in particular with the type of outcome variable.
References
1. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273:408-12.
2. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2001; 135:982-9.
3. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003; 7(1):1-68.
PDF