Considering the weight of loops for assessing the certainty of indirect evidence of network meta-analysis.

Tags: Oral
Yao L1, Guyatt G1, Brignardello-Petersen R1
1 Department of Health Research Methods, Evidence and Impact, McMaster University

Background:

Indirect evidence needs to be assessed when rating the certainty of evidence of a network meta-analysis (NMA). To increase efficiency, the current GRADE approach focuses the assessment of the indirect evidence on the dominant first order loop. This may lead, however, to ignoring an important proportion of the evidence and to inappropriate ratings of the certainty of evidence.

Objectives:

To determine the extent to which current GRADE guidance results in important errors in the assessment of certainty of the evidence.

Methods:

We will use a sample of 30 NMAs, using data from previous work conducted in the

Department of Health Research Methods, Evidence and Impact at McMaster University and research teams that collaborate with us. Using a contribution matrix, we will estimate the weight of the first order loop and other loops in the body of indirect evidence. For each comparison, we will assess the certainty of indirect evidence 1. using current GRADE guidance, and 2. considering all the loops that contribute at least 10% of the indirect evidence, until we have accounted for at least 90% of all the indirect evidence.

Results:

This is work in progress. We will present the main characteristics and the weight of the dominant first order loop of included NMAs. We will summarize the proportion of comparisons in which the certainty of the evidence differs when using current GRADE guidance versus when considering other loops in addition to the first order loop, and characterize the nature of the discrepancies. We will explore if discrepancies are related to the contribution of the first order loop to the indirect evidence.

Conclusions:

The results of this study will inform if current GRADE guidance should be revised.