An empirical assessment of minimal important differences in network meta-analysis

Article type
Authors
Da Costa B1, Saadat P2, Strauss S3, Tricco A4, Tsokani S5, Veroniki A6
1Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, Canada; Clinical Trial Service Unit and Epidemiological Studies Unit (CTSU), Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom
2Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, Canada
3Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, Canada; Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, Toronto, Canada; Department of Geriatric Medicine, University of Toronto, Toronto, Canada
4Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division & Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Canada; Queen’s Collaboration for Health Car Quality: A JBI Centre of Excellence, Kingston, Canada
5Methods Support Unit, Cochrane CET, London, United Kingdom
6Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, Canada; Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, Toronto, Canada
Abstract
Background
Systematic reviews with network meta-analysis (NMA) synthesize data from multiple studies comparing different interventions within a single framework. However, interpreting NMA findings, especially when it comes to clinical significance, can prove challenging. The minimal important difference (MID), representing the smallest clinically relevant change between two intervention groups, can facilitate interpretation of NMA evidence. MIDs are also used in the imprecision assessment of NMA results, which is part of the evaluation process of certainty of findings. CINeMA (Confidence In Network Meta-Analysis) provides a methodological framework to evaluate certainty of evidence in NMA.

Objectives
Our objective is to assess whether different clinically relevant MID thresholds impact the interpretation of NMA results and their certainty of evidence.

Methods
Categorizing by disease topic 658 NMAs of pharmacologic interventions published until July 2018, identified in our previous database of NMAs, we found that NMAs with type 2 diabetes (T2D) were predominant (N = 64). The most frequently encountered primary outcome in T2D NMAs was glycated hemoglobin A1c (HbA1c) (N = 30). To identify relevant MID thresholds, we surveyed 30 clinicians and T2D experts using Qualtrics. We supplemented recommended MID values through a literature search in PubMed until November 2023; we will also consider any MID values reported in the 30 NMAs. Using all identified MID values, we will reanalyze the T2D HbA1c NMAs. We will assess change in confidence level (ie, no concerns, some concerns, or major concerns) in imprecision and overall certainty of evidence through CINeMA.

Results
Our search identified 152 citations on MID for T2D and HbA1c, and after title-and-abstract screening 125 studies met the eligibility criteria. Currently, these are being screened for final inclusion and data extraction. Of the 30 eligible NMAs, we will reanalyze trial-data, where available, and use the reported NMA results to infer on differences in imprecision according to the collected MID values. The results of this study will be presented at the conference.

Conclusions
MID is crucial for interpretation of NMA findings and their knowledge users, including researchers, healthcare providers, and policymakers. Our results will shed light on how different MID thresholds influence NMA interpretation and certainty of evidence.