An innovative box plot for assessing data trends associated with selective dissemination of evidence

Article type
Authors
Bax L1, Wang G1, Satoh T1, Moons C2
1Kitasato Clinical Research Center, Kitasato University, Sagamihara, Japan
2Julius Center, UMC Utrecht, Utrecht, Utrecht, Netherlands
Abstract
Background: Box plots are widely used in exploratory data analyses of data from primary research. However, they are commonly not used in meta-analyses because of their inherent inability to integrate the relative weights of studies. Methods: We developed a horizontal box plot that uses the central 80% of the data points of the per-study standardized estimates (z-scores). The z-score is the effect estimate of an individual study, weighted by the inverse of the standard error. Similar to the Galbraith plot, this makes the units interpretable as standard normal deviations. Furthermore, the 0 value on the box plot s axis corresponds to the null-hypothesis, which makes it possible to integrate p-value limits in vertical shades. The individual study z-scores are shown at the bottom of the plot. Results: The plot can be used to identify data trends that are associated with the selective dissemination of evidence, e.g. missingness of studies with insignificant results. Asymmetry of the box around its central line indicates that the studies are distributed with different density around the median of the standardized effect. The plotted residuals at the bottom and the p-value shades can be used to assess whether the part of low study density coincides with high p-value (insignificant) ranges, indicating that the data trend may have been caused by selective dissemination processes. Conclusions: The proposed box plot (Figure 1) is consistent with common meta-analytical techniques and allows for straightforward assessment of data trends associated with selective dissemination of evidence.