Article type
Year
Abstract
Background: Network meta-analysis is a relatively new type of data synthesis statistics, there is still considerable concern about its validity.
Objectives: To evaluate the consistency between standard pairwise meta-analysis and network meta-analysis and explore the potential factors that are associated with the inconsistency.
Methods: PubMed, the Cochrane Library, EMBASE were searched and reference lists of relevant methodological reviews were checked to identify network meta-analyses that reported the estimate effects of both standard pairwise meta-analysis and network meta-analysis, or provided raw data to allow us to calculate direct estimates. We assessed both quantitative consistency (whether the difference between the matched estimates is statistically significant) and and qualitative consistency (whether the direction or statistical significance of the paired estimates is different). The association between inconsistency and potential influential factor was tested by the χ2 test.
Results: A total of 1901 matched estimates of standard pairwise meta-analyses and network meta-analyses were obtained from 90 studies. Of which 20 (1.1%, 95% confidence interval (CI) 0.6% to 1.5%) and 367 (19.3%, 95% CI 17.55% to 21.1%) matched estimates showed quantitative inconsistency and qualitative inconsistency, respectively. The qualitative inconsistency was significantly associated with continuous data (P value 0.002), random-effects network meta-analysis model (P value 0.045), closed loop in the network (P value 0.001), invalid coherence assumption (P value 0.036), and high risk of publication bias (P value 0.019). The quantitative and qualitative inconsistency rates for matched estimates of indirect comparisons and standard pairwise meta-analysis were 9.1% (5/55) and 32.7% (18/55), respectively.
Conclusions: Network meta-analysis is a reliable statistical procedure if it is done correctly. It hardly shows any quantitative inconsistency when compared to standard pairwise meta-analysis. However, qualitative inconsistencies exist in about one-fifth of the estimates, which should be interpreted with caution.
Objectives: To evaluate the consistency between standard pairwise meta-analysis and network meta-analysis and explore the potential factors that are associated with the inconsistency.
Methods: PubMed, the Cochrane Library, EMBASE were searched and reference lists of relevant methodological reviews were checked to identify network meta-analyses that reported the estimate effects of both standard pairwise meta-analysis and network meta-analysis, or provided raw data to allow us to calculate direct estimates. We assessed both quantitative consistency (whether the difference between the matched estimates is statistically significant) and and qualitative consistency (whether the direction or statistical significance of the paired estimates is different). The association between inconsistency and potential influential factor was tested by the χ2 test.
Results: A total of 1901 matched estimates of standard pairwise meta-analyses and network meta-analyses were obtained from 90 studies. Of which 20 (1.1%, 95% confidence interval (CI) 0.6% to 1.5%) and 367 (19.3%, 95% CI 17.55% to 21.1%) matched estimates showed quantitative inconsistency and qualitative inconsistency, respectively. The qualitative inconsistency was significantly associated with continuous data (P value 0.002), random-effects network meta-analysis model (P value 0.045), closed loop in the network (P value 0.001), invalid coherence assumption (P value 0.036), and high risk of publication bias (P value 0.019). The quantitative and qualitative inconsistency rates for matched estimates of indirect comparisons and standard pairwise meta-analysis were 9.1% (5/55) and 32.7% (18/55), respectively.
Conclusions: Network meta-analysis is a reliable statistical procedure if it is done correctly. It hardly shows any quantitative inconsistency when compared to standard pairwise meta-analysis. However, qualitative inconsistencies exist in about one-fifth of the estimates, which should be interpreted with caution.