Article type
Year
Abstract
Background: Reviews within the Cochrane Public Health and Health Systems (PHHS ) Network often involve complex interventions such as interventions that assess the effectiveness of policies, or are conducted at the population level, or in low resource settings. Therefore, the designs of trials included in these reviews often deviate from the gold standard of double-blinded, randomized controlled trials more often seen in reviews of clinical interventions.
Objectives: to:
1) assess the applicability and usability of the Cochrane 'Risk of bias 2' tool (ROB-2) to reviews within the PHHS Network;
2) assess whether the use of ROB-2 could have an impact on the GRADE ratings compared to using ratings from ROB-1;
3) identify areas where ratings are likely to be different for ROB-1 versus ROB-2.
Methods: we selected 17 studies from the most recently published reviews from each Cochrane Review Group (CRG) within the PHHS Network, searched for on 7 February 2019. Key selection criteria for study selection were trials that:
1) contributed to an outcome with a GRADE rating; and
2) were parallel-group, two-arm, individually-randomized RCTs.
Two review authors assessed the risk of bias of each study twice. The review authors started by using the ROB-1 tool to rate all studies. Once that was completed, they used ROB-2 tool to rate each study. The review authors adhered strictly to guidance from chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (2011) when using the ROB-1 tool, and the published guidance (Revised Cochrane risk-of-bias tool for randomized trials (ROB-2), edited by Julian PT Higgins et al on behalf of the ROB-2 Development Group, 15 March 2019 version) for ROB-2. Where there was insufficient information to rate the risk in ROB-1, or no clear guidance was provided in the Handbook, this was rated as 'unclear'. The reviewers recorded the time taken to complete each rating, and whether there were domains or questions in the tools which were unclear.
Results: we rated a total of 17 RCTs from six reviews. At the time of the abstract submission, the analyses of results were pending. Results presented will include a comparison of GRADE ratings for the domain of 'study limitations' between ROB-1 and ROB-2, and whether the risk of bias ratings of the corresponding domains in ROB-1 and ROB-2 are closely correlated. If there are any signalling questions or domains which are unclear, this will be noted.
Conclusions: ROB-2 assesses the risk of bias for results, rather than for outcomes or studies. Therefore, it has potential to have a large impact on the assessment of the risk of bias for reviews on prevention and policy interventions. This could have implications for GRADE, and we will describe how ROB-2 affects the GRADE assessments of the six different reviews assessed, and whether there are any domains/signalling questions that need to be clarified in order to make this more applicable to PHHS reviews.
Patient or healthcare consumer involvement: none
Objectives: to:
1) assess the applicability and usability of the Cochrane 'Risk of bias 2' tool (ROB-2) to reviews within the PHHS Network;
2) assess whether the use of ROB-2 could have an impact on the GRADE ratings compared to using ratings from ROB-1;
3) identify areas where ratings are likely to be different for ROB-1 versus ROB-2.
Methods: we selected 17 studies from the most recently published reviews from each Cochrane Review Group (CRG) within the PHHS Network, searched for on 7 February 2019. Key selection criteria for study selection were trials that:
1) contributed to an outcome with a GRADE rating; and
2) were parallel-group, two-arm, individually-randomized RCTs.
Two review authors assessed the risk of bias of each study twice. The review authors started by using the ROB-1 tool to rate all studies. Once that was completed, they used ROB-2 tool to rate each study. The review authors adhered strictly to guidance from chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (2011) when using the ROB-1 tool, and the published guidance (Revised Cochrane risk-of-bias tool for randomized trials (ROB-2), edited by Julian PT Higgins et al on behalf of the ROB-2 Development Group, 15 March 2019 version) for ROB-2. Where there was insufficient information to rate the risk in ROB-1, or no clear guidance was provided in the Handbook, this was rated as 'unclear'. The reviewers recorded the time taken to complete each rating, and whether there were domains or questions in the tools which were unclear.
Results: we rated a total of 17 RCTs from six reviews. At the time of the abstract submission, the analyses of results were pending. Results presented will include a comparison of GRADE ratings for the domain of 'study limitations' between ROB-1 and ROB-2, and whether the risk of bias ratings of the corresponding domains in ROB-1 and ROB-2 are closely correlated. If there are any signalling questions or domains which are unclear, this will be noted.
Conclusions: ROB-2 assesses the risk of bias for results, rather than for outcomes or studies. Therefore, it has potential to have a large impact on the assessment of the risk of bias for reviews on prevention and policy interventions. This could have implications for GRADE, and we will describe how ROB-2 affects the GRADE assessments of the six different reviews assessed, and whether there are any domains/signalling questions that need to be clarified in order to make this more applicable to PHHS reviews.
Patient or healthcare consumer involvement: none