Article type
Abstract
Background: Network meta-analysis (NMA) extends conventional meta-analysis to allow simultaneous comparisons between multiple interventions. NMA is well-established in biomedical research and now increasingly used and advocated as a way to synthesise evidence on multiple alternative complex social interventions.
Objectives: To investigate the feasibility, challenges, risks and benefits of NMA as a research synthesis method for comparing multiple complex social interventions.
Methods: Two NMA applications were compared using different strategies and datasets. The first application defined a network of prison-based drug treatments using clinical categories and then used multivariate design-by-treatment interaction NMA with subgroup analyses. The second defined an evidence network of probation and parole programmes using intervention component combinations and used Bayesian NMA.
Results: Over 50 studies were included in each review but the large number of interventions meant that both networks were sparse. Networks were also unbalanced and characterised by high levels of heterogeneity. Poor primary study reporting quality, including missing information about intervention designs, implementation and population characteristics, raised additional challenges. Statistical assessments yielded no evidence of inconsistency in either network but were underpowered in both.
Conclusions: A multiple treatment comparison and network perspective ensured that high-quality, relevant and previously-excluded evidence was taken into account. Gaps in the evidence base and problems with treatment rankings were more clearly identified than in previous pairwise meta-analyses. The utility of NMA for social interventions was, however, limited by network (1) sparseness (2) imbalance; and, (3) poor reporting. Little confidence could be placed on NMA results. Adherence to recently improved reporting guidelines; consensus approaches to intervention definitions; and primary research on complex social intervention mechanisms and evidence network features are needed to ensure that the potential benefits of NMA are realised and misleading conclusions avoided.
Objectives: To investigate the feasibility, challenges, risks and benefits of NMA as a research synthesis method for comparing multiple complex social interventions.
Methods: Two NMA applications were compared using different strategies and datasets. The first application defined a network of prison-based drug treatments using clinical categories and then used multivariate design-by-treatment interaction NMA with subgroup analyses. The second defined an evidence network of probation and parole programmes using intervention component combinations and used Bayesian NMA.
Results: Over 50 studies were included in each review but the large number of interventions meant that both networks were sparse. Networks were also unbalanced and characterised by high levels of heterogeneity. Poor primary study reporting quality, including missing information about intervention designs, implementation and population characteristics, raised additional challenges. Statistical assessments yielded no evidence of inconsistency in either network but were underpowered in both.
Conclusions: A multiple treatment comparison and network perspective ensured that high-quality, relevant and previously-excluded evidence was taken into account. Gaps in the evidence base and problems with treatment rankings were more clearly identified than in previous pairwise meta-analyses. The utility of NMA for social interventions was, however, limited by network (1) sparseness (2) imbalance; and, (3) poor reporting. Little confidence could be placed on NMA results. Adherence to recently improved reporting guidelines; consensus approaches to intervention definitions; and primary research on complex social intervention mechanisms and evidence network features are needed to ensure that the potential benefits of NMA are realised and misleading conclusions avoided.