Graphical augmentations to the funnel plot for assessing the impact of additional evidence on a meta-analysis

Article type
Authors
Langan D1, Higgins J2, Gregory W1, Crowther M3, Sutton A3
1Clinical Trials Research Unit, University of Leeds, UK
2MRC Biostatistics Unit, Cambridge, UK
3University of Leicester, UK
Abstract
Background: Methods to quantify the potential impact of future evidence on a systematic review are currently under-developed but would greatly aid decisions on both review updating priorities and recommendations for future research.

Objectives: Develop novel overlays to the funnel plot to provide a visual illustration of the impact that a new study would have on a given meta-analysis. The additional features help inform: i) the current robustness of a meta-analysis; ii) sample size calculations for the design of future studies to be added to the meta-analysis; and iii) the update prioritisation strategy for a portfolio of meta-analyses (such as those managed by Cochrane Review Groups).

Description of developed methodology: Several inter-related overlays to the funnel plot are described and illustrated using data from Cochrane reviews. These include: i) statistical significance contours, which define regions of the funnel plot in which a new study would have to be located in order to change the statistical significance of the meta-analysis (an example of these for a sleep apnoea review is included in the figure); and ii) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. The use of the aforementioned overlays simultaneously with further features including: pooled treatment effects, lines of no effect, confidence and prediction intervals, simulated new studies and sample size guidelines, are all discussed. Details of free software for creating the plots in both R and STATA are provided. Suggestions for how review editors, review authors and trial designers could use the plots will be considered.

Conclusions: The authors believe the developed enhancements to the funnel plot provide a lot of accessible information about the impact of future evidence on an existing evidence base. It is hoped editors, systematic reviewer authors and trialists will find such plots useful.