Background: The prevalence and impact of outcome reporting bias (ORB) in trials on Cochrane reviews has been examined [1]. ORB was suspected in at least one trial in 34% of 283 Cochrane reviews. However, in this situation trials may report other outcomes that are correlated with the missing outcome of interest; one can then utilise a multivariate meta-analysis to jointly synthesise all the reported outcomes and ‘borrow strength’ across them [2]. This approach has the potential to reduce the impact of ORB. Objectives: (i) To compare the performance of the multivariate meta-analysis approach compared to the univariate approach; (ii) To determine whether the ‘‘borrowing of strength’’ in a multivariate meta-analysis can reduce the impact of ORB, and by how much. Methods: A simulation study was conducted considering the fixed effect bivariate meta-analysis setting. In each simulation, the bias and coverage of the pooled meta-analysis result was assessed. Data were simulated for situations where a) both outcomes were reported in all studies, and b) outcome data were informatively missing. Results: Results show that the ‘‘borrowing of strength’’ in a multivariate meta-analysis can reduce the magnitude of the impact of ORB. Bias, mean-square error and coverage are improved using the multivariate meta-analysis approach, with improvements increasing as the correlation between outcomes increased. Conclusions: The reliability of meta-analysis can be improved if more attention is paid to missing outcome data. If a high percentage of data is missing, reviewers should be encouraged to contact the trialists to confirm whether an outcome was measured and analysed and, if so, to obtain the results. If this is not possible, a multivariate meta-analysis approach is a potential statistical solution for reducing the impact of ORB.
References
1. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ (2010); 340:c356.
2. Riley RD. Multivariate meta-analysis: the effect of ignoring within study correlation. J. R. Statist. Soc. A (2009); 172 (4) 789–811.