A forward search algorithm for detection of extreme study effects in network meta-analysis

Tags: Oral
Petropoulou M1, Salanti G2, Rücker G3, Schwarzer G3, Moustaki I4, Mavridis D1
1Department of Primary Education, University of Ioannina, 2Institute of Social and Preventive Medicine (ISPM), University of Bern, 3Institute of Medical Biometry and Statistics, Medical Center-University of Freiburg, 4Department of Statistics, London School of Economics

Background: In a quantitative synthesis of studies via meta-analysis, it is likely that some studies provide a markedly different intervention effect or have a large impact on effect estimates and/or heterogeneity. There are several methods to identify outlying studies in a meta-analysis. Most of them rely on backward methods that evaluate the impact that a study has on the meta-analysis results once it has been removed. It has been proven that such methods have a high likelihood of false positives and false negative results. One robust outlying diagnostic method developed initially for regression analysis that has been applied to meta-analysis is the forward search (FS) algorithm. Little work has been done in the field of network meta-analysis (NMA), which allows for multiple treatment comparisons and a joint synthesis of direct and indirect evidence. In the NMA setting, extreme study effects may be contribute to statistical inconsistency; that is the disagreement between direct and indirect evidence.

Objectives: To extend the FS algorithm to NMA and create a relevant R library.

Methods: FS algorithm starts by fitting the NMA model to a small subset of studies and then, gradually adding studies according to their closeness to the hypothesized model until all studies are included. Outlying and influential cases are detected through a series of forward plots of statistics related to model parameters and fit. We extended each stage of the FS algorithm to NMA and we developed a routine in R. We applied the suggested outlier detection method to real and artificial, contaminated by outliers, networks to explore their performance.

Results: Application in real networks indicated that studies responsible for heterogeneity and inconsistency influenced the results.

Conclusions: The FS algorithm is very useful in detecting outlying studies as well as studies responsible for a large heterogeneity and inconsistency. FS algorithm is a promising diagnostic tool for dealing with heterogeneity and inconsistency. Outliers can be excluded in a sensitivity analysis to explore robustness of results.

Patient or healthcare consumer involvement: None.