Article type
Year
Abstract
Background: Cochrane intervention reviews rely primarily on evidence from randomized trials (RCTs), but this is not always available. Some clinical questions such as those about rare, long term or unexpected harms, are often addressed incompletely by RCTs. There is no widely accepted tool for assessing risk of bias in non-randomized studies (NRS) of interventions.
Objectives: Describe key methodological concepts that underpin development of the new Cochrane tool for assessing risk of bias in NRS.
Methods: We developed the tool through a wide consultation with methodologists, authors, Cochrane Review Groups and other stakeholders. Methodological considerations were addressed primarily through discussions within domain-focused working groups and a face-to-face meeting of all contributors.
Results: Evaluations of risk of bias in the results of NRS are facilitated by considering each study as an attempt to mimic a hypothetical RCT comparing the same interventions as the NRS (the ‘target trial’) and by specifying the intervention effect of interest (either assignment to the interventions at baseline, or receiving the interventions as specified in the protocol). Bias domains can be divided into “pre-intervention” domains (confounding and selection bias), an “at intervention” domain (bias in measuring interventions), and “post-intervention” domains (bias due to departures from intended interventions, missing data, measurement of outcomes and selection of the reported result). Assessment of pre-intervention bias domains is mainly distinct from assessments in RCTs (these focus on randomization procedures), but there is substantial overlap in assessment of post-intervention domains in RCTs and NRS. Studies that split follow-up according to intervention received require special consideration because confounding can occur at baseline and at the time of intervention change.
Conclusions: Evaluating risk of bias in a systematic review of NRS requires both methodological and content expertise. A structured approach to assessing risk of bias in different domains should provide a systematic way to organize and present the available evidence relating to risk of bias.
Objectives: Describe key methodological concepts that underpin development of the new Cochrane tool for assessing risk of bias in NRS.
Methods: We developed the tool through a wide consultation with methodologists, authors, Cochrane Review Groups and other stakeholders. Methodological considerations were addressed primarily through discussions within domain-focused working groups and a face-to-face meeting of all contributors.
Results: Evaluations of risk of bias in the results of NRS are facilitated by considering each study as an attempt to mimic a hypothetical RCT comparing the same interventions as the NRS (the ‘target trial’) and by specifying the intervention effect of interest (either assignment to the interventions at baseline, or receiving the interventions as specified in the protocol). Bias domains can be divided into “pre-intervention” domains (confounding and selection bias), an “at intervention” domain (bias in measuring interventions), and “post-intervention” domains (bias due to departures from intended interventions, missing data, measurement of outcomes and selection of the reported result). Assessment of pre-intervention bias domains is mainly distinct from assessments in RCTs (these focus on randomization procedures), but there is substantial overlap in assessment of post-intervention domains in RCTs and NRS. Studies that split follow-up according to intervention received require special consideration because confounding can occur at baseline and at the time of intervention change.
Conclusions: Evaluating risk of bias in a systematic review of NRS requires both methodological and content expertise. A structured approach to assessing risk of bias in different domains should provide a systematic way to organize and present the available evidence relating to risk of bias.