Reasons or excuses for avoiding meta-analysis in forest plots

Article type
Authors
Patsopoulos N, Ioannidis J, Rothstein H
Abstract
Background: Some systematic reviews assemble the eligible studies without performing meta-analysis. This may be a legitimate choice. However, an interesting situation arises when reviews present forest plots, but do not calculate a summary. These imply that it is important to visualize the quantitative data, but final synthesis is inappropriate. Objectives: To present reasons given by researchers for avoiding final synthesis in forest plots and discuss analytical options in these cases. Methods: We assessed systematic reviews (SRs) that included at least one forest plot with >= 2 studies in the Cochrane Database of Systematic Reviews (CDSR, 4, 2005). We selected SRs that presented forest plots but no summary estimate. For forest plots with >= 4 studies and no summary effect due to high statistical heterogeneity, we also calculated I² and respective confidence intervals (CIs). Results: Of the 1739 SRs that included at least one forest plot with >= 2 studies in the CDSR, 135 reviews (8%) had 559 forest plots with X2 studies but no summary estimate. The reasons given were: different interventions compared (n = 41/30%), too high statistical heterogeneity (36/27%), different metrics or outcomes evaluated (25/19%), different study designs (21/16%), different study participants/settings (21/16%), synthesis considered inappropriate (not specified why) (15/11%), too limited data (11/8%), no reason given (10/7%), clinical heterogeneity (not otherwise specified) (9/7%), data with many counts per participant (5/4%), and non-normality of data (1/1%). In almost all cases, analytical options existed for possible quantitative synthesis. In 22 forest plots with >= 4 studies where a synthesis was avoided because statistical heterogeneity was considered too high, I² ranged between 35% and 98% with median of 71%. The lower 95% CI of I² was <25% in 11 of the 22 non-summarized forest plots. Conclusions: One of the advantages of meta-analysis is to assess, examine and/or model the consistency of effects, and improve understanding of moderator variables, boundary conditions, and generalizability. Different patients and different studies are unavoidably heterogeneous. This diversity and the uncertainty associated with it should be explored whenever possible.