Statistical simulation to assess results of meta-analyses using post-intervention, change from baseline and mixed methods

Article type
Authors
Silva V1, Grande AJ1, Porfírio GJM1, Puga MES1, Martimbianco ALC1, Torres MFS1, Costa MB1, Carvalho MR1, Fioretti BTDS1, Costa CS1, Peccin MS1, Riera R1, Torloni MR1, Atallah A1
1Brazilian Cochrane Centre, BR
Abstract
Background: Meta-analyses of continuous outcomes can be performed by post-intervention, changes from baseline and also by a mixture of these methods (Fig. 1a). General recommendations are available in the Cochrane Handbook for Systematic Reviews of Interventions in chapters 7 (Section 7.7.3.1), 9 (Section 9.4.5.2) and 16 (Section 16.1.3.2), however, the impact of the choice of a methods has not been assessed.

Objective: To assess the impact in estimated effect (effect size and confidence interval) in meta-analyses performed by post-intervention, change from baseline and by a mixture of these methods.

Methods: We produced 20 theoretical randomized controlled trials (RCT) (10 small-trials and 10 mega-trials) through statistical simulation using Software R (version 2.15.3) (Fig. 1b). The function was developed based in a random number generation for the normal distribution. Continuous outcome data were produced; mean and standard deviation were calculated for the experimental and control groups of the post-intervention and change from baseline from each RCT. The theoretical RCTs analyzed by post-intervention and change from baseline were randomized for mixture of methods. Random-effects meta-analyses of mean difference and standardized mean difference were performed, followed by the 95% confidence interval, using the inverse-variance method. Sensitivity analyses were conducted.

Results: Differences in the effect sizes ranged from 0.10 to 9.01 and differences in the width of the confidence intervals ranged from 1.16 to 18.04 (Fig. 2 and Table 1).

Conclusions: Despite the lack of significant differences in this statistical simulation, there were changes in the effect sizes and confidence intervals indicating that meta-analyses using post-intervention, change from baseline and mixture of methods can produce different conclusions especially if the effect estimated is close to accepting or rejecting the null hypothesis. Sensitivity analyses are recommended when the methods were mixed.