Article type
Year
Abstract
Background: Network meta-analyses (NMAs) provide comparative evidence about the effects of all healthcare interventions for a given condition. However, NMA results can be undermined by flaws in design, conduct, analyses and reporting. Several tools are available for the critical appraisal of an NMA. However, none aim specifically to assess the risk of bias (RoB) in an NMA.
Objectives: To describe a new RoB NMA tool to assess the risk of bias in an NMA, developed using rigorous methodology.
Methods: As an international steering committee, we developed a protocol and made conceptual decisions about the tool’s structure. We developed the tool using a multistep approach: definition of the scope of the tool; conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs [1]; web-based Delphi survey to collate expert opinion regarding the quality of NMAs [2]; an online survey to solicit decision-makers’ needs [2]; and refinement through piloting.
Results: The RoB NMA tool includes three domains: bias arising from the network characteristics/geometry, bias arising from potential effect modifiers, and bias arising from statistical synthesis. The three domains comprise a total of 16 statements which can identify potential limitations of the NMA, leading to a judgement about the risk of bias within each domain (low, high or unclear risk of bias). The tool should be used to identify potential limitations in the way an NMA was conducted that may lead to bias in its results or conclusions. The RoB NMA tool should be conducted as a supplement to the assessment of the risk of bias in the underlying evidence review, for example, using the ROBIS tool. Piloting of the RoB NMA tool is ongoing.
Conclusions: The development of the RoB NMA tool was evidence based and included piloting and refinement. We believe that the RoB NMA tool will allow objective rating of the RoB in an NMA.
Patient, public and/or healthcare consumer involvement: None.
Objectives: To describe a new RoB NMA tool to assess the risk of bias in an NMA, developed using rigorous methodology.
Methods: As an international steering committee, we developed a protocol and made conceptual decisions about the tool’s structure. We developed the tool using a multistep approach: definition of the scope of the tool; conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs [1]; web-based Delphi survey to collate expert opinion regarding the quality of NMAs [2]; an online survey to solicit decision-makers’ needs [2]; and refinement through piloting.
Results: The RoB NMA tool includes three domains: bias arising from the network characteristics/geometry, bias arising from potential effect modifiers, and bias arising from statistical synthesis. The three domains comprise a total of 16 statements which can identify potential limitations of the NMA, leading to a judgement about the risk of bias within each domain (low, high or unclear risk of bias). The tool should be used to identify potential limitations in the way an NMA was conducted that may lead to bias in its results or conclusions. The RoB NMA tool should be conducted as a supplement to the assessment of the risk of bias in the underlying evidence review, for example, using the ROBIS tool. Piloting of the RoB NMA tool is ongoing.
Conclusions: The development of the RoB NMA tool was evidence based and included piloting and refinement. We believe that the RoB NMA tool will allow objective rating of the RoB in an NMA.
Patient, public and/or healthcare consumer involvement: None.