Background: Evidence about how much medical interventions work may change over time. We aimed to determine what fluctuations in the treatment effect reported by clinical trials and their meta-analyses may be expected and whether extreme fluctuations signal future major changes.

Methods: We applied recursive cumulative meta-analysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 medical interventions in 2 medical disciplines (pregnancy/perinatal medicine n=45 interventions; myocardial infarction n=15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes > 2.5 standard deviations of the expected.

Results: With 500 accumulated patients, the pooled odds ratio may change by 0.6 to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74 and 1.35-fold for pregnancy/perinatal medicine and between 0.83 and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of pre-eclampsia) where recent mega-trials have contradicted prior meta-analyses as well as in 4 other examples where early large treatment effects were dissipated when more data appeared.

Conclusions: Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their meta-analyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may signal further major changes in the future.