How to improve the credibility of meta-analysis? How to avoid mistakes in choosing the statistical method of estimation? The Bayesian and traditional approaches

Article type
Authors
Nikodem M1, Walczak J1, Borowiack E2, Siedmiogrodzki K1, Zapalska A2, Khan K3, Meads C4, Mol B5, Oude Rengerink K5, Thangaratinam S3, Zamora J6
1CASPolska Association, Poland
2Arcana Institute, Poland
3Queen Mary, University of London, Great Britain
4Queen Mary, University of London
5Academic Medical Center Amsterdam, Netherlands
6Hospital Universitario Ramon y Cajal, Madrid, Spain
Abstract
Background: There are at least ten different statistical methods of meta-analysis. It can be surprising how different results of estimation can be obtained just by applying different method. Therefore, if we want to run the meta-analysis properly then we have to solve the crucial problem of choosing the most credible statistical method. This is especially important in the case of small sample size, low event rate or large discrepancy between event rates in groups of comparison.

Objectives: To find out which methods are the most credible and precise for particular data type. Moreover to underline which methods should not be used to avoid substantial errors in estimation. Our main aim is to answer the question of how to deal with following problems: a. insufficient sample size, b. low event rate.

Methods: Initially a systematic review on existing statistical methods was conducted. It was followed by an expanded analysis of credibility and precision of statistical methods. Ten statistical methods (including alternative methods such as Bayesian ones combining results of nonrandomized with randomized trials) were examined on different data sets. Overall 124 sub-cases (different sample size, event rate, heterogeneity, etc.) were considered. Bias, confidence interval coverage and statistical power were the measures of credibility and precision.

Results: Possible datasets were categorized and for particular categories the optimal methods are highlighted and warnings of possible mistakes are underlined. All results are presented in the form of guidelines.

Conclusions: There is no universal method which is credible for every set of data, therefore the choice of method should be done carefully and depending on the data characteristics. For several cases, the Bayesian approach (with or without inclusion of non-randomized trials) was highly ranked, hence it should be taken under greater consideration.