Article type
Year
Abstract
Background: Assessing the risk of bias (RoB) of included studies and then incorporating this information into the GRADE assessment is one of the most complicated aspects of conducting a review. This is especially true for reviews of non-randomized studies (NRS).
Objectives: To develop a useful tool for rating RoB in included studies that (1) clarifies which RoB items to use for both RCTs and NRS with a control group and (2) organizes the ratings by bias category to help reviewers make a decision about downgrading the evidence due to study limitations.
Methods: We used Excel to create a spreadsheet for conducting RoB for both RCTs and NRSwith a control group. Wetook RoB items from the Back Group’s RoB tool and the Downs & Black checklist and organized them by bias category (i.e., performance, attrition, measurement, and selective outcome reporting). Users are instructed to make a rating of ‘Bias’ or ‘No Bias’ within each category. These ratings are automatically copied into outcome-specific tables for ease of making judgements about downgrading the evidence. We sent a draft of the spreadsheet to colleagues within the Collaboration for evaluation.
Results: Six authors and three editors provided feedback. Respondents agreed that the tool is useful, especially the way it links RoB to GRADE. Three respondents expressed concern about having to re-enter the same information into our spreadsheet, RevMan, and GRADEpro. There were suggestions for making it more user-friendly; including the use of drop-down menus to minimize user errors.
Conclusions: A simplified version of our tool may be useful to new authors and authors working with NRS, and it may be especially valuable for training purposes. The next steps will be to communicate our findings to the Informational Management System team, conduct further validation work, and consider adapting the tool for use on smartphones.
Objectives: To develop a useful tool for rating RoB in included studies that (1) clarifies which RoB items to use for both RCTs and NRS with a control group and (2) organizes the ratings by bias category to help reviewers make a decision about downgrading the evidence due to study limitations.
Methods: We used Excel to create a spreadsheet for conducting RoB for both RCTs and NRSwith a control group. Wetook RoB items from the Back Group’s RoB tool and the Downs & Black checklist and organized them by bias category (i.e., performance, attrition, measurement, and selective outcome reporting). Users are instructed to make a rating of ‘Bias’ or ‘No Bias’ within each category. These ratings are automatically copied into outcome-specific tables for ease of making judgements about downgrading the evidence. We sent a draft of the spreadsheet to colleagues within the Collaboration for evaluation.
Results: Six authors and three editors provided feedback. Respondents agreed that the tool is useful, especially the way it links RoB to GRADE. Three respondents expressed concern about having to re-enter the same information into our spreadsheet, RevMan, and GRADEpro. There were suggestions for making it more user-friendly; including the use of drop-down menus to minimize user errors.
Conclusions: A simplified version of our tool may be useful to new authors and authors working with NRS, and it may be especially valuable for training purposes. The next steps will be to communicate our findings to the Informational Management System team, conduct further validation work, and consider adapting the tool for use on smartphones.