Exclusion of trials with no events in both arms from meta-analyses can change the conclusions

Oral
2020 Abstracts
Xu C1, Li L1, Lin L2, Chu H3, Lehana T4, Zou K5, Sun X1
1Chinese Evidence-Based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu
2Department of Statistics, Florida State University, Tallahassee
3Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis
4Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton; Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare
5Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu
Abstract

Objectives:

Classical meta-analyses routinely treat trials with no events in both arms non-informative and exclude them from analyses. This study assessed whether such trials are statistically meaningful and have any influence on conclusion of meta-analyses.
Design and setting: We collected meta-analyses of binary outcomes with at least one trial having no events in both arms from Cochrane systematic reviews published between 2003 and 2018. We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), p-value, and the width of 95% confidence interval (CI) were compared. A simulation study was further conducted to examine the robustness of results.

Results:

We identified 442 meta-analyses. In comparing the meta-analyses that included studies with no events in both arms versus those not, the flipping of direction occurred in eight (1.80%) comparisons; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on p-value (55.66% increased, 44.12% decreased) and the width of 95% CI (50.68% inflated, 49.32% declined) when excluding studies with no events. The simulation study confirmed these findings

Conclusions:

Studies with no events in both arms are not necessarily non-informative. Excluding such studies may alter conclusions.