Article type
Year
Abstract
Background: The Cochrane Collaboration’s risk of bias tool for critically evaluating randomised trials currently addresses selection bias through scrutiny of randomisation methods (sequence generation and allocation concealment). Assessment of baseline imbalances in demographic and clinical characteristics—particularly those known to be prognostically important—can indicate whether the randomisation methods successfully achieved comparability across the randomised groups. Such assessments are however not recommended in the Handbook, and when performed are seldom linked explicitly with selection bias.
Proposal: An assessment of baseline imbalances between groups should form a key and prominent part of the selection bias domain of the Cochrane risk of bias tool. To inform this, important prognostic factors, and the magnitude of the difference between groups that would be sufficient to raise concern, should be pre-specified in the review protocol. Considering baseline imbalance in addition to randomisation methods will allow many ‘Unclear’ risk judgements—which are currently made frequently—to be resolved into ‘Low’ or ‘High’ risk judgements. ‘Low’ and ‘High’ risk judgments may also be re-classified accordingly.
Rationale for proposal: During recent systematic reviews we recognised that failure to assess baseline imbalance can lead to review conclusions being either unnecessarily conservative or over-optimistic. Methods of randomisation and allocation concealment are often poorly described in published trial reports, even when it transpires (from other sources) that robust methods have been used. In contrast, baseline data tables are presented in a large majority of trial reports.
Conclusions: Our suggestions to enhance future versions of the Cochrane risk of bias tool have the potential to reduce uncertainty in systematic review conclusions, and to reduce the risk of chance findings being ascribed to treatment effects. Furthermore, they may enable better use of available evidence by a more considered approach to evaluating studies which use imperfect randomisation and allocation methods.
Proposal: An assessment of baseline imbalances between groups should form a key and prominent part of the selection bias domain of the Cochrane risk of bias tool. To inform this, important prognostic factors, and the magnitude of the difference between groups that would be sufficient to raise concern, should be pre-specified in the review protocol. Considering baseline imbalance in addition to randomisation methods will allow many ‘Unclear’ risk judgements—which are currently made frequently—to be resolved into ‘Low’ or ‘High’ risk judgements. ‘Low’ and ‘High’ risk judgments may also be re-classified accordingly.
Rationale for proposal: During recent systematic reviews we recognised that failure to assess baseline imbalance can lead to review conclusions being either unnecessarily conservative or over-optimistic. Methods of randomisation and allocation concealment are often poorly described in published trial reports, even when it transpires (from other sources) that robust methods have been used. In contrast, baseline data tables are presented in a large majority of trial reports.
Conclusions: Our suggestions to enhance future versions of the Cochrane risk of bias tool have the potential to reduce uncertainty in systematic review conclusions, and to reduce the risk of chance findings being ascribed to treatment effects. Furthermore, they may enable better use of available evidence by a more considered approach to evaluating studies which use imperfect randomisation and allocation methods.