Article type
Year
Abstract
Background: Few validated checklist tools are available to assess the risk of bias (RoB) of non-randomized studies (NRS), although systematic reviews including NRS have increased in number, for many reasons.
Objectives: The aims of the study are to develop and validate a new RoB tool for NRS.
Methods: After deciding principles and reviewing relevant tools, we developed the Risk of Bias Assessment tool for Non-randomized Studies (RoBANS) tool. Subsequent testing involved the assessment of inter-rater agreement between reviewers, time to apply the RoBANS tool, degree of correlation for overall risk compared with overall quality scores, user-convenience, and content validity.
Results: The RoBANS contains 6 domains including the selection of participants, confounding variables, measurement of intervention (exposure), blinding of outcome assessment, incomplete outcome data and selective outcome reporting (table). Inter-rater agreement of the RoBANS tool varied in individual domains (weighted κ=0.38 to 0.74). The mean time to complete the RoBANS tool was less than for the MINORS (Methodological Items for NOn-Randomized Studies) (9.5 minutes (SD 3.39) per study vs. 10.45 (SD 3.54),( P = 0.18). There was a moderate correlation between overall RoB compared with the MINORS (Kendall’s τ=0.384). Reviewers evaluated RoBANS as 'good’ in terms of user-convenience for 3 items (mean 5.7 of 7 as the Likert scale; 0.81 %). External experts (n = 8) evaluated RoBANS as 'fair’ in terms of face validity for 7 items (mean 5.4 of 7 as the Likert scale) and all recommended its use to assess the RoB for NRS.
Conclusions: RoBANS is a valid tool designed to assess the RoB of NRS. Since RoBANS is harmonized with the Cochrane’s RoB tool and GRADE (Grading of Recommendations Assessment, Development and Evaluation), and can be incorporated into RevMan and GRADEpro, it appears to be useful to people undertaking systematic reviews.
Objectives: The aims of the study are to develop and validate a new RoB tool for NRS.
Methods: After deciding principles and reviewing relevant tools, we developed the Risk of Bias Assessment tool for Non-randomized Studies (RoBANS) tool. Subsequent testing involved the assessment of inter-rater agreement between reviewers, time to apply the RoBANS tool, degree of correlation for overall risk compared with overall quality scores, user-convenience, and content validity.
Results: The RoBANS contains 6 domains including the selection of participants, confounding variables, measurement of intervention (exposure), blinding of outcome assessment, incomplete outcome data and selective outcome reporting (table). Inter-rater agreement of the RoBANS tool varied in individual domains (weighted κ=0.38 to 0.74). The mean time to complete the RoBANS tool was less than for the MINORS (Methodological Items for NOn-Randomized Studies) (9.5 minutes (SD 3.39) per study vs. 10.45 (SD 3.54),( P = 0.18). There was a moderate correlation between overall RoB compared with the MINORS (Kendall’s τ=0.384). Reviewers evaluated RoBANS as 'good’ in terms of user-convenience for 3 items (mean 5.7 of 7 as the Likert scale; 0.81 %). External experts (n = 8) evaluated RoBANS as 'fair’ in terms of face validity for 7 items (mean 5.4 of 7 as the Likert scale) and all recommended its use to assess the RoB for NRS.
Conclusions: RoBANS is a valid tool designed to assess the RoB of NRS. Since RoBANS is harmonized with the Cochrane’s RoB tool and GRADE (Grading of Recommendations Assessment, Development and Evaluation), and can be incorporated into RevMan and GRADEpro, it appears to be useful to people undertaking systematic reviews.
Images
