Article type
Abstract
Background
Two approaches are available to assess the certainty of the evidence (CoE) of network meta-analysis (NMA) results: the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the Confidence in Network Meta-Analysis (CINeMA). Although they share many common aspects, their operationalization differs.
Objectives
To compare the 2 approaches in terms of similarities of results, interrater agreement among assessors and time needed to apply each tool.
Methods
This is a randomized controlled study. The 2 approaches will be applied to networks of 2 dichotomous and 2 continuous outcomes with different network structure and size, from 4 NMAs of randomized controlled trials (RCTs) on: disease-modifying therapies for relapsing multiple sclerosis, antiplatelet treatments for secondary stroke prevention, pharmacological treatment for seizures in adults, and pharmacological and surgical treatments for people with obesity. We will evaluate only treatments versus 1 common comparator. Thirteen assessors who never applied both approaches but have been trained on NMA methodology and such methods will be randomly assigned to 4 groups applying GRADE and CINeMA in different order to different outcomes. See Figure 1.
We will evaluate:
-Number of discrepant CoE ratings and magnitude of the difference for each comparison in the networks.
-Interrater reliability (Fleiss’ Kappa) for multiple raters for each domain and overall judgments.
-Time to complete the assessment in minutes for each network with the 2 approaches.
Results
The network on continuous outcome “cognitive decline” included 6 RCTs, 7 treatments, and no loops versus interferon beta 1a. The network on “weight loss in Kg,” continuous, included 60 RCTs, 8 treatments, and 5 loops versus lifestyle modification. The network on “seizure response,” dichotomous, included 16 RCTs, 6 treatments, and no loop versus placebo. The network on dichotomous outcome “ischemic stroke” included 38 RCTs, 15 treatments, and 4 loops versus placebo. See Figure 2.
Conclusions
Conclusions will be based on our findings.
Relevance and importance to patients
By improving efficiency and consistency in assessing the CoE in NMAs, our findings will contribute to improving the quality of the evidence base for decision-making.
Patient and public involvement
Not applicable.
Two approaches are available to assess the certainty of the evidence (CoE) of network meta-analysis (NMA) results: the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the Confidence in Network Meta-Analysis (CINeMA). Although they share many common aspects, their operationalization differs.
Objectives
To compare the 2 approaches in terms of similarities of results, interrater agreement among assessors and time needed to apply each tool.
Methods
This is a randomized controlled study. The 2 approaches will be applied to networks of 2 dichotomous and 2 continuous outcomes with different network structure and size, from 4 NMAs of randomized controlled trials (RCTs) on: disease-modifying therapies for relapsing multiple sclerosis, antiplatelet treatments for secondary stroke prevention, pharmacological treatment for seizures in adults, and pharmacological and surgical treatments for people with obesity. We will evaluate only treatments versus 1 common comparator. Thirteen assessors who never applied both approaches but have been trained on NMA methodology and such methods will be randomly assigned to 4 groups applying GRADE and CINeMA in different order to different outcomes. See Figure 1.
We will evaluate:
-Number of discrepant CoE ratings and magnitude of the difference for each comparison in the networks.
-Interrater reliability (Fleiss’ Kappa) for multiple raters for each domain and overall judgments.
-Time to complete the assessment in minutes for each network with the 2 approaches.
Results
The network on continuous outcome “cognitive decline” included 6 RCTs, 7 treatments, and no loops versus interferon beta 1a. The network on “weight loss in Kg,” continuous, included 60 RCTs, 8 treatments, and 5 loops versus lifestyle modification. The network on “seizure response,” dichotomous, included 16 RCTs, 6 treatments, and no loop versus placebo. The network on dichotomous outcome “ischemic stroke” included 38 RCTs, 15 treatments, and 4 loops versus placebo. See Figure 2.
Conclusions
Conclusions will be based on our findings.
Relevance and importance to patients
By improving efficiency and consistency in assessing the CoE in NMAs, our findings will contribute to improving the quality of the evidence base for decision-making.
Patient and public involvement
Not applicable.