Article type
Year
Abstract
Background:
Systematic reviews (SRs) that collate data about the relative effects of multiple interventions via network meta-analysis (NMA) are highly informative for decision-making purposes. A NMA provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions.
Objectives:
To propose an approach based on the methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses to determining confidence in the output of a NMA. Five domains are judged: study limitations, indirectness, inconsistency, imprecision and publication bias.
Methods:
The suggested framework for evaluating a NMA acknowledges (1) the key role of indirect comparisons (2) the contributions of each piece of direct evidence to the NMA estimates of effect size; (3) the importance of the transitivity assumption to the validity of NMA; and (4) the possibility of disagreement between direct evidence and indirect evidence. We applied the strategy to a SR comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations.
Results:
The suggestions proposed were workable in the empirical example. Out of seven effect estimates, two had moderate confidence, four low and one very low confidence mainly due to study limitations, inconsistency and imprecision. Our confidence in the ranking of the included treatments was low.
Conclusions:
The proposed framework can be used to determine confidence in the results from a NMA. The assumption of transitivity which is key to a NMA should be considered in the indirectness domain and its assessment is critical. In the inconsistency component, both between-study variance within a comparison and differences between direct and indirect evidence in the network should be evaluated. The contributions of each piece of direct evidence can make the judgments less subjective.
Systematic reviews (SRs) that collate data about the relative effects of multiple interventions via network meta-analysis (NMA) are highly informative for decision-making purposes. A NMA provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions.
Objectives:
To propose an approach based on the methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses to determining confidence in the output of a NMA. Five domains are judged: study limitations, indirectness, inconsistency, imprecision and publication bias.
Methods:
The suggested framework for evaluating a NMA acknowledges (1) the key role of indirect comparisons (2) the contributions of each piece of direct evidence to the NMA estimates of effect size; (3) the importance of the transitivity assumption to the validity of NMA; and (4) the possibility of disagreement between direct evidence and indirect evidence. We applied the strategy to a SR comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations.
Results:
The suggestions proposed were workable in the empirical example. Out of seven effect estimates, two had moderate confidence, four low and one very low confidence mainly due to study limitations, inconsistency and imprecision. Our confidence in the ranking of the included treatments was low.
Conclusions:
The proposed framework can be used to determine confidence in the results from a NMA. The assumption of transitivity which is key to a NMA should be considered in the indirectness domain and its assessment is critical. In the inconsistency component, both between-study variance within a comparison and differences between direct and indirect evidence in the network should be evaluated. The contributions of each piece of direct evidence can make the judgments less subjective.