Article type
Year
Abstract
Background: Network meta-analysis (NMA) is rapidly reaching the forefront of health care research. To avoid misleading conclusions and provide valuable information for clinical decision making, NMAs need to be assessed thoroughly against potential sources of bias. A potentially major threat of the validity of results from NMA are studies with markedly different or extreme effect sizes, namely outlying studies. Such studies could substantially influence and bias the conclusions of the NMAs and thus need proper investigation. Whilst several outlier detection methods have been developed for standard meta-analysis, little work has been done in the field of NMA.
Objectives: Our aim is to propose an intuitive Bayesian model that captures deviating studies within a network of interventions and to explore the influence of such studies in the NMA results under different modelling scenarios.
Methods: We define outliers as studies with ‘shifted’ effect sizes and based on this, we introduce a Bayesian NMA mean-shifted model, which assumes shifted effects sizes for each study. Then, we use Bayes factors to test whether each study is more compatible to the conventional or to the mean-shifted NMA model. In the latter situation the study is considered as an outlier. Furthermore, detection of outliers is not straightforward when there is a cluster of outliers. To mitigate this issue, we embed the whole procedure in a leave-one-out cross validation scheme where we restrict our search to groups of studies comparing either the same treatments or the same class of treatments.
Results: We explored the performance of our method using simulated networks contaminated by artificial outliers and a real network of 112 randomized controlled trials comparing second-line treatments for advanced non-small cell lung cancer. The method successfully identified existing outliers in the simulated data. In the lung cancer network, we identified one clear and two potential outliers corresponding to a very large and two moderate Bayes factors respectively. The impact and influence of each of these studies has been assessed via contribution matrices and sensitivity analysis. In both cases, results suggest that two out of the three potential outliers are affecting the results, with one study being particularly influential.
Conclusions: Our method offers an effective diagnostic tool for the identification of outlying and influential studies in a network of interventions. Sensitivity analysis is used to exclude outliers and assess result robustness. This has the clear potential to avoid inappropriate NMA conclusions while aiding robust clinical judgments and correct interpretation of results.
Patient or healthcare consumer involvement: None.
Objectives: Our aim is to propose an intuitive Bayesian model that captures deviating studies within a network of interventions and to explore the influence of such studies in the NMA results under different modelling scenarios.
Methods: We define outliers as studies with ‘shifted’ effect sizes and based on this, we introduce a Bayesian NMA mean-shifted model, which assumes shifted effects sizes for each study. Then, we use Bayes factors to test whether each study is more compatible to the conventional or to the mean-shifted NMA model. In the latter situation the study is considered as an outlier. Furthermore, detection of outliers is not straightforward when there is a cluster of outliers. To mitigate this issue, we embed the whole procedure in a leave-one-out cross validation scheme where we restrict our search to groups of studies comparing either the same treatments or the same class of treatments.
Results: We explored the performance of our method using simulated networks contaminated by artificial outliers and a real network of 112 randomized controlled trials comparing second-line treatments for advanced non-small cell lung cancer. The method successfully identified existing outliers in the simulated data. In the lung cancer network, we identified one clear and two potential outliers corresponding to a very large and two moderate Bayes factors respectively. The impact and influence of each of these studies has been assessed via contribution matrices and sensitivity analysis. In both cases, results suggest that two out of the three potential outliers are affecting the results, with one study being particularly influential.
Conclusions: Our method offers an effective diagnostic tool for the identification of outlying and influential studies in a network of interventions. Sensitivity analysis is used to exclude outliers and assess result robustness. This has the clear potential to avoid inappropriate NMA conclusions while aiding robust clinical judgments and correct interpretation of results.
Patient or healthcare consumer involvement: None.