Article type
Year
Abstract
Background: according to the traditional GRADE approach, indirect comparisons should usually lead to lowering the quality of the evidence due to indirectness by at least one point. Therefore, network meta-analyses (NMA) that combine direct and indirect comparisons would be expected to downgrade their ratings for that reason. However, proposals to better fit GRADE to NMA are reluctant to downgrade indirect evidence by default.
Objectives: to describe the certainty of the evidence for the main comparisons in the published Cochrane NMA. Also, to analyze the reasons for downgrading certainty according to GRADE domains, especially indirectness. To compare the certainty of the evidence obtained from the mixed (direct and indirect) comparisons with that obtained from the direct comparisons of origin. To evaluate the consistency between the different NMA on presenting the 'Summary of findings' table.
Methods: on 15 April 2019 we carried out a search on the Cochrane Library website using 'Network Meta-Analysis' as a MeSH term to identify all published Cochrane NMA. We retrieved all the comparisons included in the 'Summary of findings' tables of the identified reviews in order to analyze the application of the GRADE criteria for evaluating the certainty of the evidence.
Results: we retrieved 20 NMA, published between 2016 and 2018. We excluded four reviews since they did not finally conduct a NMA. We evaluated a total of 254 comparisons. The certainty of the NMA evidence was high in 9.1% of the comparisons, moderate in 22.4%, low in 27.6% and very low in 17.3%. There were no data for 23.6% of all the analyzed comparisons.
The reasons for downgrading the certainty of the evidence were: imprecision (42.8%), risk of bias (24.1%), indirectness (21.0%), inconsistency (8.2%), others (3.9%). Reasons for downgrading certainty were not provided in a review. Only two NMAs (10%) systematically reduced certainty due to indirectness in all the comparisons. None of the NMAs reduced uncertainty by two levels due to indirectness. Only three of all the NMAs (15%) showed certainty from direct evidence, from indirect evidence and certainty for the NMA separately in the 'Summary of findings' table. We identified four comparisons in which the certainty of the evidence was graded higher for the NMA than for the direct evidence of origin.
Conclusions: in most of the cases certainty of the NMA evidence was not systematically downgraded due to indirect comparisons. Indirectness was hardly ever considered as a reason for downgrading the certainty of the evidence.
A high heterogeneity exists between the NMA when evaluating the certainty of the evidence and also in presenting the results. Very few NMA reported results from both direct and indirect evidence separately. Certainty of the evidence for a NMA should not be higher than that for the direct comparisons of origin.
Patient or healthcare consumer involvement: none
Objectives: to describe the certainty of the evidence for the main comparisons in the published Cochrane NMA. Also, to analyze the reasons for downgrading certainty according to GRADE domains, especially indirectness. To compare the certainty of the evidence obtained from the mixed (direct and indirect) comparisons with that obtained from the direct comparisons of origin. To evaluate the consistency between the different NMA on presenting the 'Summary of findings' table.
Methods: on 15 April 2019 we carried out a search on the Cochrane Library website using 'Network Meta-Analysis' as a MeSH term to identify all published Cochrane NMA. We retrieved all the comparisons included in the 'Summary of findings' tables of the identified reviews in order to analyze the application of the GRADE criteria for evaluating the certainty of the evidence.
Results: we retrieved 20 NMA, published between 2016 and 2018. We excluded four reviews since they did not finally conduct a NMA. We evaluated a total of 254 comparisons. The certainty of the NMA evidence was high in 9.1% of the comparisons, moderate in 22.4%, low in 27.6% and very low in 17.3%. There were no data for 23.6% of all the analyzed comparisons.
The reasons for downgrading the certainty of the evidence were: imprecision (42.8%), risk of bias (24.1%), indirectness (21.0%), inconsistency (8.2%), others (3.9%). Reasons for downgrading certainty were not provided in a review. Only two NMAs (10%) systematically reduced certainty due to indirectness in all the comparisons. None of the NMAs reduced uncertainty by two levels due to indirectness. Only three of all the NMAs (15%) showed certainty from direct evidence, from indirect evidence and certainty for the NMA separately in the 'Summary of findings' table. We identified four comparisons in which the certainty of the evidence was graded higher for the NMA than for the direct evidence of origin.
Conclusions: in most of the cases certainty of the NMA evidence was not systematically downgraded due to indirect comparisons. Indirectness was hardly ever considered as a reason for downgrading the certainty of the evidence.
A high heterogeneity exists between the NMA when evaluating the certainty of the evidence and also in presenting the results. Very few NMA reported results from both direct and indirect evidence separately. Certainty of the evidence for a NMA should not be higher than that for the direct comparisons of origin.
Patient or healthcare consumer involvement: none
PDF