Incorporating uncertainty of between-trial variance estimates increases the reliability of random-effects meta-analysis

Article type
Authors
Thorlund K, Wetterslev J, Gluud C
Abstract
Background: In spite of its theoretical merits in a meta-analysis context, the DerSimonian-Laird random-effects model is often criticized for being overly sensitive to various sources of bias. This problem stems from the inverse variance weighting method, which gives similar weights to the included trials if there is considerable heterogeneity. Further, trials may appear deceivingly homogenous in the presence of some time dependent type of bias (e.g. time lag bias or publication bias). Consequently, the total variation may be underestimated and result in too narrow confidence intervals. These two shortcomings may be fixed by incorporating uncertainty of the between-trial variance estimate in the inverse variance weighting method. Objectives: To conduct a comprehensive simulation study to compare the reliability of the standard random-effects model meta-analysis, originally proposed by DerSimonian and Laird, to randomeffects model meta-analysis that incorporates uncertainty of the betweentrial variance estimate in the inverse variance weighting method. Methods: We simulated a comprehensive set of meta-analysis scenarios varying magnitude of treatment effect, control group event rate, heterogeneity, trial size distribution, and presence of time dependent type of bias. We measured bias of pooled summary effect estimates and coverage of 95% confidence intervals for k = 2, y,30 trials. Results: Incorporating uncertainty of the between-trial variance estimate typically reduced bias of pooled summary effect estimates with 5% to15% compared to the traditional DerSimonian-Laird random-effects model meta-analysis. Meta-analysis incorporating uncertainty of the betweenstudy variance estimate retained coverage close to the desired 95% when the simulated data was subject to a time dependent type of bias. The DerSimonian-Laird random-effects model consistently provided coverage lower than 85%. Both approaches attained non-biased results when the simulated data was unbiased. Conclusions: Incorporating uncertainty of the between-trial variance estimate in a random-effects model increases the reliability of meta-analysis. The method should become standard in meta-analytic practice.