Article type
Abstract
Background: Network meta-analysis (NMA) plays a crucial role in clinical decision-making by comparing multiple interventions simultaneously, enhancing statistical power through the integration of direct and indirect evidence, and ranking interventions. Despite their advantages, many NMAs rely on Surface Under the Cumulative Ranking curve (SUCRA) or P-scores for intervention ranking without considering the certainty of evidence. This practice may elevate interventions with low certainty to top ranks, leading to potentially misleading conclusions with adverse implications for patient care.
Objectives: To quantify the prevalence of potentially misleading conclusions in NMAs published in top medical journals, and to propose recommendations for enhancing the appropriate consideration of evidence certainty in NMA practices.
Methods: This study conducted a comprehensive review of NMAs from leading medical journals from 2019 to February 2024, with an impact factor greater than 30. Our study focused on publications that provided intervention rankings and the certainty of evidence for each intervention. Our analysis identified NMAs that concluded in favor of interventions underpinned by low-certainty evidence, including evidence categorized as low to very low certainty.
Results: We identified 45 NMAs, of which 28 (62%) used SUCRA, and 15 (33%) used P-score for ranking of interventions. 23 (51%) used CINeMA, and 22 (49%) used GRADE for rating the certainty of evidence. Seven (16%) NMAs ranked the interventions based on moderate to high certainty of evidence, whereas a majority presented low or very low certainty evidence in their rankings. We are still analyzing how often the top-ranked interventions were based on low or very low certainty evidence. The preliminary findings indicate that the issue of drawing misleadingly optimistic conclusions based on low certainty evidence from NMAs is not uncommon in prestigious medical journals.
Conclusions: This study highlights the importance of appropriately considering the certainty during the ranking of interventions for NMAs comparing multiple interventions.
Objectives: To quantify the prevalence of potentially misleading conclusions in NMAs published in top medical journals, and to propose recommendations for enhancing the appropriate consideration of evidence certainty in NMA practices.
Methods: This study conducted a comprehensive review of NMAs from leading medical journals from 2019 to February 2024, with an impact factor greater than 30. Our study focused on publications that provided intervention rankings and the certainty of evidence for each intervention. Our analysis identified NMAs that concluded in favor of interventions underpinned by low-certainty evidence, including evidence categorized as low to very low certainty.
Results: We identified 45 NMAs, of which 28 (62%) used SUCRA, and 15 (33%) used P-score for ranking of interventions. 23 (51%) used CINeMA, and 22 (49%) used GRADE for rating the certainty of evidence. Seven (16%) NMAs ranked the interventions based on moderate to high certainty of evidence, whereas a majority presented low or very low certainty evidence in their rankings. We are still analyzing how often the top-ranked interventions were based on low or very low certainty evidence. The preliminary findings indicate that the issue of drawing misleadingly optimistic conclusions based on low certainty evidence from NMAs is not uncommon in prestigious medical journals.
Conclusions: This study highlights the importance of appropriately considering the certainty during the ranking of interventions for NMAs comparing multiple interventions.