Assessing the risk of confounding and selection bias in a systematic review that includes non-randomized studies (NRS)

Tags: Workshop
Reeves B1, Shea B1, Wells G2, Sterne J3, Higgins J3, Savovic J3
1Non-Randomised Studies Methods Group, 2University of Ottawa Heart Institute, Canada, 3University of Bristol, United Kingdom


The workshop aims to train review authors to assess the risk of bias due to confounding and selection of participants into groups when including non-randomized studies (NRS) in systematic reviews about the effects of interventions, using an extended risk of bias tool (ACROBAT-NRS).


Cochrane recommends that review authors consider and justify whether to include NRS for all research questions. Decisions to include NRS may arise when there are inadequate, or no randomized controlled trials (RCTs), but where the question addressed by the review is a considered a priority. Review questions about possible harmful or long-term effects of interventions, or about the effects of public health and non-pharmacological interventions, may have these characteristics.

This workshop aims to give review authors and others experience in applying ACROBAT-NRS. Participants will work in small groups to apply the confounding and selection bias domains of the tool, which do not apply to RCTs. Signalling questions prompt the user to assess key aspects of studies, and then to judge whether a study is at high or low risk of material bias in these domains for specified outcomes. Responses to signalling questions and domain-specific bias judgements are made on four-point scales and include a ‘no information’ option; these features will be contrasted with the existing Cochrane risk of bias tool. Check the programme for two other workshops on aspects of ACROBAT-NRS.