Background: Network meta-analysis is an important extension of pairwise meta-analysis to compare more than two interventions and to analyze indirect and direct evidence simultaneously. It is becoming more and more popular in systematic reviews and health technology assessment. Although for pairwise meta-analysis the properties of the different approaches are well examined, little is known for the approaches in network meta-analysis.
Objectives: To compare methods that are currently available for estimating and assessing inconsistency in network meta-analysis.
Methods: A simulation study was conducted to evaluate complex networks with up to five interventions and different patterns. We analyzed the impact of different network-sizes, different amounts of inconsistency and heterogeneity on MSE and coverage of established and new approaches. This exceeds previous simulations studies, which have been conducted by others.
Methods: We found that, with a high degree of inconsistency in the network, none of the evaluated effect estimators produced reliable results. For a network with no or just moderate inconsistency the Bayesian and the frequentist estimator showed acceptable properties, whereas the latter one showed slightly better results. We also found a dependency on the amount of heterogeneity in the network.
Conclusions: Our results underline the need to assess inconsistency in network meta-analyses reliably as available measures for inconsistency may be misleading in many situations. We therefore conclude that it is also important to assess similarity (regarding population, intervention, etc.) and heterogeneity to reduce inconsistency in the network in advance. Nevertheless, effect estimators should be used which are suitable in the case of moderate inconsistency.