Article type
Year
Abstract
BACKGROUND:As per the traditional GRADE recommendations, indirect comparisons should normally imply lowering the quality of the evidence due to indirectness at least 1 point. Hence, network meta-analyses (NMA) that mix direct and indirect comparisons would be expected to downgrade their scores for that matter but new approaches to better match GRADE to NMA do not consider downgrading indirect evidence by default
OBJECTIVES:To outline the quality of the evidence presented in the Summary of Findings Tables (SFT) from the published Cochrane NMA. Furthermore, to examine the rationale for lowering the certainty by GRADE domains. To contrast the certainty from the combined (direct and indirect) comparisons with that obtained from the direct comparisons. To assess the degree of agreement among the different NMA when reporting the SFT and how transitivity and incoherence/inconsistency are evaluated
METHODS:A search in the Cochrane Library website using “Network Meta-Analysis” as a MeSH term and within titles, abstracts or keywords was conducted on March 13, 2020 to locate all published Cochrane NMA. All the comparisons listed in the SFT of the identified reviews were collected in order to evaluate the implementation of the GRADE criteria for assessing the quality of the evidence
RESULTS:Forty-one NMA were recovered, published between January 2016 and March 2020. Fourteen reviews were excluded due to lack of reporting of at least one indirect or combined comparison in SFT format. A total of 859 comparisons were evaluated. The certainty of the NMA evidence was “high” in 11.1% of the combined comparisons, “moderate” in 18.3%, “low” in 21.8% and “very low” in 22.4%. There were no data for 26.4% of all the analyzed comparisons. The reasons for downgrading the certainty of the evidence were: imprecision (49.6%), risk of bias (30.1%), indirectness (10.7%), inconsistency/incoherence (7.5%), others (2.1%). One review did not report any reason for downgrading the certainty regarding combined comparisons. Only in four NMA (14.8%) quality was systematically lessened due to indirectness and none was reduced two levels due to this reason. In seven of the 27 NMA (26%) certainty can be explored separately from direct and combined evidence in the SFT. In sixteen comparisons (11%), certainty of the evidence was graded higher for the NMA respect to direct evidence. Only in twenty-two comparisons (15%) direct evidence was graded higher. No explicit assessment of transitivity and incoherence/inconsistency was found in 7 (26%) and 1 (4%) reviews respectively
CONCLUSIONS:In general, quality of the NMA evidence was not routinely downgraded due to indirect comparisons. Indirectness was rarely considered as a reason for lowering the quality of the evidence. There is great heterogeneity among NMA in assessing the certainty of the evidence and also in showing the results. Only a few NMA published results from direct, indirect and combined evidence separately. Transitivity was not thoroughly explored in some reviews
Patient or healthcare consumer involvement:None
OBJECTIVES:To outline the quality of the evidence presented in the Summary of Findings Tables (SFT) from the published Cochrane NMA. Furthermore, to examine the rationale for lowering the certainty by GRADE domains. To contrast the certainty from the combined (direct and indirect) comparisons with that obtained from the direct comparisons. To assess the degree of agreement among the different NMA when reporting the SFT and how transitivity and incoherence/inconsistency are evaluated
METHODS:A search in the Cochrane Library website using “Network Meta-Analysis” as a MeSH term and within titles, abstracts or keywords was conducted on March 13, 2020 to locate all published Cochrane NMA. All the comparisons listed in the SFT of the identified reviews were collected in order to evaluate the implementation of the GRADE criteria for assessing the quality of the evidence
RESULTS:Forty-one NMA were recovered, published between January 2016 and March 2020. Fourteen reviews were excluded due to lack of reporting of at least one indirect or combined comparison in SFT format. A total of 859 comparisons were evaluated. The certainty of the NMA evidence was “high” in 11.1% of the combined comparisons, “moderate” in 18.3%, “low” in 21.8% and “very low” in 22.4%. There were no data for 26.4% of all the analyzed comparisons. The reasons for downgrading the certainty of the evidence were: imprecision (49.6%), risk of bias (30.1%), indirectness (10.7%), inconsistency/incoherence (7.5%), others (2.1%). One review did not report any reason for downgrading the certainty regarding combined comparisons. Only in four NMA (14.8%) quality was systematically lessened due to indirectness and none was reduced two levels due to this reason. In seven of the 27 NMA (26%) certainty can be explored separately from direct and combined evidence in the SFT. In sixteen comparisons (11%), certainty of the evidence was graded higher for the NMA respect to direct evidence. Only in twenty-two comparisons (15%) direct evidence was graded higher. No explicit assessment of transitivity and incoherence/inconsistency was found in 7 (26%) and 1 (4%) reviews respectively
CONCLUSIONS:In general, quality of the NMA evidence was not routinely downgraded due to indirect comparisons. Indirectness was rarely considered as a reason for lowering the quality of the evidence. There is great heterogeneity among NMA in assessing the certainty of the evidence and also in showing the results. Only a few NMA published results from direct, indirect and combined evidence separately. Transitivity was not thoroughly explored in some reviews
Patient or healthcare consumer involvement:None