Article type
Year
Abstract
Background: Systematic reviews help inform policy and healthcare decisions. Randomised trials are the optimal study design for evaluating the effectiveness of interventions. Random sequence generation is used to help minimise selection bias in the way participants are allocated to groups. We have observed that some reviewers judge randomised trials as at high or unclear risk of bias (ROB) for sequence generation, but the extent of this issue has not been well described.
Objectives: We evaluated the consistency in the ROB assessment for sequence generation for randomised trials in Cochrane and non-Cochrane reviews.
Methods: We retrieved Cochrane intervention reviews (1 Jan 2017 to 31 Mar 2017) from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in 10 general medical journals with the highest impact factors (1 Jan 2016 to 31 Mar 2017). We examined the proportion of reviews that rated the sequence generation domain as at high, low or unclear risk of selection bias. For reviews that had rated any randomised trials at high or unclear risk of bias, we examined the proportion that assessed the quality of evidence.
Results: We included 105 Cochrane and 85 non-Cochrane reviews. Of the 1352 randomised trials included across the Cochrane Reviews, 1% were rated at high and 46% at unclear ROB for sequence generation. For non-Cochrane reviews, these figures were 4% and 44%, respectively. The 65% of non-Cochrane reviews that had rated randomised trials at high ROB for sequence generation had not reported why. All Cochrane Reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only two-thirds had assessed the quality of evidence.
Conclusions: Systematic reviews of interventions frequently rate randomised trials at high or unclear ROB for sequence generation. Consistent ROB assessment for sequence generation contributes to minimise selection bias, and transparent quality of evidence assessment ensure that conclusions are linked to the data. In general, Cochrane reviews were more transparent than non-Cochrane reviews on ROB and quality of evidence judgements. Standardised reporting of trial characteristics should be strongly promoted to improve evaluation of evidence.
Patient or healthcare consumer involvement: No patients or consumers were directly involved in the conduct of this study.
Objectives: We evaluated the consistency in the ROB assessment for sequence generation for randomised trials in Cochrane and non-Cochrane reviews.
Methods: We retrieved Cochrane intervention reviews (1 Jan 2017 to 31 Mar 2017) from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in 10 general medical journals with the highest impact factors (1 Jan 2016 to 31 Mar 2017). We examined the proportion of reviews that rated the sequence generation domain as at high, low or unclear risk of selection bias. For reviews that had rated any randomised trials at high or unclear risk of bias, we examined the proportion that assessed the quality of evidence.
Results: We included 105 Cochrane and 85 non-Cochrane reviews. Of the 1352 randomised trials included across the Cochrane Reviews, 1% were rated at high and 46% at unclear ROB for sequence generation. For non-Cochrane reviews, these figures were 4% and 44%, respectively. The 65% of non-Cochrane reviews that had rated randomised trials at high ROB for sequence generation had not reported why. All Cochrane Reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only two-thirds had assessed the quality of evidence.
Conclusions: Systematic reviews of interventions frequently rate randomised trials at high or unclear ROB for sequence generation. Consistent ROB assessment for sequence generation contributes to minimise selection bias, and transparent quality of evidence assessment ensure that conclusions are linked to the data. In general, Cochrane reviews were more transparent than non-Cochrane reviews on ROB and quality of evidence judgements. Standardised reporting of trial characteristics should be strongly promoted to improve evaluation of evidence.
Patient or healthcare consumer involvement: No patients or consumers were directly involved in the conduct of this study.