Article type
Year
Abstract
Background: Cochrane intervention reviews rely primarily on evidence from randomized trials (RCTs), but this is not always available. Some clinical questions, such as those about rare, long term or unexpected harms, are often addressed incompletely by RCTs. There is currently no widely accepted tool for assessing risk of bias in non-randomized studies (NRS) of interventions.
Objectives: To present the new Cochrane tool for assessing the risk of bias in NRS.
Methods: We developed the tool through consultations with methodologists, authors, Cochrane Review Groups and other stakeholders. We used a combination of interactive and iterative methods including: a scope meeting with key stakeholders; survey of review groups’ needs; recruitment of experts; small domain-focused working groups; and a meeting with all contributors. The draft tool was revised following piloting.
Results: Protocol considerations include specification of a target trial, the effect of interest, and likely confounding domains and co-interventions. The tool consists of seven bias domains: confounding; selection of participants into the study; measurement of interventions; departures from intended interventions; missing data; measurement of outcomes; and selection of the reported result. Each domain includes 3 to 5 signalling questions that focus on key aspects of study conduct. Answers to these questions inform a domain-level judgment on risk of bias that has five levels: low risk, when a study is comparable to a well-performed RCT; moderate risk, consistent with a sound NRS but not comparable to an RCT; Serious risk, when a study has some important problems; Critical risk, when a study is too problematic to provide useful evidence; and No information. Criteria for reaching a judgment about the overall risk of bias are specified.
Conclusions: This is the first domain-based risk of bias tool for NRS of interventions, developed with wide international participation of methodological experts and stakeholders. It provides a structured approach to organizing and presenting available evidence relating to risk of bias and will be disseminated across the Collaboration after the Hyderabad Colloquium.
Objectives: To present the new Cochrane tool for assessing the risk of bias in NRS.
Methods: We developed the tool through consultations with methodologists, authors, Cochrane Review Groups and other stakeholders. We used a combination of interactive and iterative methods including: a scope meeting with key stakeholders; survey of review groups’ needs; recruitment of experts; small domain-focused working groups; and a meeting with all contributors. The draft tool was revised following piloting.
Results: Protocol considerations include specification of a target trial, the effect of interest, and likely confounding domains and co-interventions. The tool consists of seven bias domains: confounding; selection of participants into the study; measurement of interventions; departures from intended interventions; missing data; measurement of outcomes; and selection of the reported result. Each domain includes 3 to 5 signalling questions that focus on key aspects of study conduct. Answers to these questions inform a domain-level judgment on risk of bias that has five levels: low risk, when a study is comparable to a well-performed RCT; moderate risk, consistent with a sound NRS but not comparable to an RCT; Serious risk, when a study has some important problems; Critical risk, when a study is too problematic to provide useful evidence; and No information. Criteria for reaching a judgment about the overall risk of bias are specified.
Conclusions: This is the first domain-based risk of bias tool for NRS of interventions, developed with wide international participation of methodological experts and stakeholders. It provides a structured approach to organizing and presenting available evidence relating to risk of bias and will be disseminated across the Collaboration after the Hyderabad Colloquium.