Article type
Year
Abstract
Background: Meta-analyses often only include a few studies. Estimating between-study heterogeneity is difficult in these cases. An inaccurate estimation of heterogeneity can result in biased effect estimates and confidence intervals (CIs) that are too narrow. Research has shown that this is particularly true when using the DerSimonian and Laird (DLRE) heterogeneity variance estimator.
Methods: To compare different methods for meta-analysis of a small number of studies with binary outcomes.
Results: We compared the DLRE method with other meta-analytic methods, including the modified Hartung-Knapp (mHK) method, the Paule-Mandel (PM) method and the beta-binominal (BB) model considering odds ratios. For the comparison of the methods for meta-analysis of few studies (≤ 5), we performed a simulation study that used true parameters from meta-analyses that had actually been performed in Cochrane Reviews to mirror meta-analyses observed in practice. For each scenario we simulated 10,000 meta-analyses. We performed various sensitivity analyses to assess the robustness of the results (effect sizes, baseline probabilities, between-study heterogeneity, 10 to 50 studies in meta-analysis). We estimated median bias, 95% empirical coverage, power and robustness to assess the performance of the methods.
Conclusions: All methods outperformed the standard DLRE method. The BB model performed best, considering the balance between correct empirical coverage and power for meta-analysis of few studies. If one is willing to accept a slightly higher type-I error rate than the nominal level for a higher power, the PM methods is an alternative.
Patient or healthcare consumer involvement: Not applicable.
Methods: To compare different methods for meta-analysis of a small number of studies with binary outcomes.
Results: We compared the DLRE method with other meta-analytic methods, including the modified Hartung-Knapp (mHK) method, the Paule-Mandel (PM) method and the beta-binominal (BB) model considering odds ratios. For the comparison of the methods for meta-analysis of few studies (≤ 5), we performed a simulation study that used true parameters from meta-analyses that had actually been performed in Cochrane Reviews to mirror meta-analyses observed in practice. For each scenario we simulated 10,000 meta-analyses. We performed various sensitivity analyses to assess the robustness of the results (effect sizes, baseline probabilities, between-study heterogeneity, 10 to 50 studies in meta-analysis). We estimated median bias, 95% empirical coverage, power and robustness to assess the performance of the methods.
Conclusions: All methods outperformed the standard DLRE method. The BB model performed best, considering the balance between correct empirical coverage and power for meta-analysis of few studies. If one is willing to accept a slightly higher type-I error rate than the nominal level for a higher power, the PM methods is an alternative.
Patient or healthcare consumer involvement: Not applicable.