Article type
Year
Abstract
Background: In randomised trials, continuous data are often measured before and after the intervention. Common analytical methods include analysing final values (FV), analysing change scores (CS) and analysis of covariance (ANCOVA). These approaches provide unbiased estimates of the same true underlying intervention effect. Theoretically, pooling effects from a mix of these methods should yield unbiased estimates of pooled intervention effect. However, the meta-analytic estimates may be biased if the trialists selectively report results.
Objectives: To investigate if the pooled estimate of intervention effect and its standard error were modified by the analytical methods employed in the component trials.
Methods: Fifty-four meta-analyses were selected from The Cochrane Database of Systematic Reviews (2004, Issue 4). Twenty met the inclusion criteria of at least four comparisons from parallel group trials with enough data available to extract at least one effect estimate, with a mix of analytical methods across the comparisons. Pooling effects using a mix FV and CS were compared with: pooling effects from i) one analytical method (either all FV or all CS), ii) a mix of FV, CS, and ANCOVA, iii) all ANCOVA. A 'meta-meta-analysis’ was undertaken to investigate if the pooled effect was modified by the use of either FV or CS analyses.
Results: Analyses are suggestive of selective reporting of analytical method by baseline imbalance. However, the impact of this on the random effects meta-analytical estimates was less clear. Inconsistency of effects (I2) varied: greater consistency was observed with approach i) and greater inconsistency with approach iii). Standard errors, p-values, and between trial heterogeneity were similar between the approaches.
Conclusions: Pooling effects from a mix of analytical methods is appealing, however it may be problematic. The inclusion of ANCOVA estimates did not result in expected gains in terms of increased precision of the meta-analytic estimates.
Objectives: To investigate if the pooled estimate of intervention effect and its standard error were modified by the analytical methods employed in the component trials.
Methods: Fifty-four meta-analyses were selected from The Cochrane Database of Systematic Reviews (2004, Issue 4). Twenty met the inclusion criteria of at least four comparisons from parallel group trials with enough data available to extract at least one effect estimate, with a mix of analytical methods across the comparisons. Pooling effects using a mix FV and CS were compared with: pooling effects from i) one analytical method (either all FV or all CS), ii) a mix of FV, CS, and ANCOVA, iii) all ANCOVA. A 'meta-meta-analysis’ was undertaken to investigate if the pooled effect was modified by the use of either FV or CS analyses.
Results: Analyses are suggestive of selective reporting of analytical method by baseline imbalance. However, the impact of this on the random effects meta-analytical estimates was less clear. Inconsistency of effects (I2) varied: greater consistency was observed with approach i) and greater inconsistency with approach iii). Standard errors, p-values, and between trial heterogeneity were similar between the approaches.
Conclusions: Pooling effects from a mix of analytical methods is appealing, however it may be problematic. The inclusion of ANCOVA estimates did not result in expected gains in terms of increased precision of the meta-analytic estimates.