Methodological review of items for assessing the risk of bias in network meta-analyses provides groundwork for the creation of an extension to the ROBIS tool

Article type
Authors
Lunny C1, Andrea T2, Wright J1, Whiting P3
1Cochrane Hypertension Group, University of British Columbia
2University of Toronto
3University of Bristol
Abstract
Background: To decide the best treatment for a patient, healthcare providers and patients need a synthesis of evidence for all possible treatments for a given condition. Network meta-analysis (NMA) emerged due to the limitations of pairwise meta-analyses to provide comparative effectiveness of multiple treatments for the same condition. Conventional meta-analyses only average the randomised trials comparing two treatments. NMA can help patients and their care providers choose the treatment that is most important to them based on side effects and efficacy of all treatments.
Tools are available for most study designs to make quality assessment easier for a knowledge user. For example, ROBIS can be used to assess the risk of bias of systematic reviews (SRs). However, there is currently no risk of bias tool for network meta-analyses (NMA). As the main ROBIS domains are applicable to systematic reviewsSRs, ROBIS’ synthesis domain can be adapted as an extension for assessing NMAs.
Objectives: To conduct a methodologicalscoping review with the aim to develop a list of items relating to risk of bias in network meta-analyses.
Methods: We searched MEDLINE, Embase, Cochrane library as well as grey literature databases including the EQUATOR Network, websites of evidence synthesis organisations (Cochrane, the US Institute of Medicine, the Campbell Collaboration, and the Joanna Briggs Institute), as well as methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program). We included any article describing or reporting items related to risk of bias in NMAs. We also included studies that assessed the methodological quality of NMAs. To identify other potentially relevant studies, we examined the reference lists of included studies and undertook forward citation searches of seminal articles using Google Scholar. Two reviewers independently reviewed titles, abstracts and full text articles. Data was extracted on items, criteria or guidance that was potentially relevant to the risk of bias or quality of NMAs of interventions. Sources were ordered and extracted by year of publication, and when a new source was reviewed, items already extracted were revised iteratively or added if they were unique. The final list of items deemed unique (i.e. same conceptual or methodological issue) were retained. The items were categorised based on the synthesis domain in the ROBIS structure.
Results: A list of items was developed and categorised into broad themes based on the ROBIS tool. When items related to the same conceptual or methodological issue they were combined, and other unique concepts were split into separate items. Many items were reworded as signalling questions so that each item is phrased so “yes” is good, and each item only covers a single concept.
Conclusions: This review provides groundwork for the creation of an extension to the ROBIS tool for assessing risk of bias in NMAs. Knowledge users need the highest quality evidence to make decisions about which treatments should be used in healthcare.