Article type
Year
Abstract
Background: The Cochrane 'Risk of bias tool (CRoB) is one of the most widely used tools for assessing the risk of bias (RoB) of clinical trials. However, there are no clear, detailed guidelines for its application and its poor inter-rater reliability (IRR) has been a wide concern.
Objectives: To develop a framework (iCRoB) as a users’ guide and to improve the IRR of the CRoB in its first 4 domains.
Methods:
1. Develop a step-by-step structured pathway for assessing the RoB.
2. Identify and summarize possible scenarios that are used to describe a domain in clinical trials.
3. Merge the identified scenarios with those already provided in the CRoB. The bias assessment pathway and the new dictionary of scenarios in combination are the components from the iCRoB.
4. Conduct a randomized controlled study to compare IRR among individual raters and that across rater pairs between CRoB and iCRoB.
Results: We designed a structured pathway for assessing bias systematically, which helps classify a study into one of five categories for each RoB domain. A total of 34, 36, 26 and 20 scenarios were generated for sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment, respectively. Trial results showed that the iCRoB had a higher IRR across rater pairs than the original CRoB for every domain. The weighted κ was 0.71 and 0.81 for sequence generation respectively for CRoB and iCRoB; 0.53 and 0.61 for allocation concealment respectively for CRoB and iCRoB; 0.56 for blinding of participants and personnel in CRoB, 0.68 for blinding of participants and 0.70 for blinding of personnel in iCRoB; and 0.19 and 0.43 for blinding of outcome assessment respectively for CRoB and iCRoB.
Conclusions: We developed the iCRoB for making the judgement on RoB in reports of clinical trials. Our iCRoB showed a higher reliability than the current CRoB in all the domains examined. The iCRoB can be improved further by new contributions to the dictionary of scenarios and made easy by automating the bias assessment. Importantly, the iCRoB has created a possibility for automating the bias assessment.
Objectives: To develop a framework (iCRoB) as a users’ guide and to improve the IRR of the CRoB in its first 4 domains.
Methods:
1. Develop a step-by-step structured pathway for assessing the RoB.
2. Identify and summarize possible scenarios that are used to describe a domain in clinical trials.
3. Merge the identified scenarios with those already provided in the CRoB. The bias assessment pathway and the new dictionary of scenarios in combination are the components from the iCRoB.
4. Conduct a randomized controlled study to compare IRR among individual raters and that across rater pairs between CRoB and iCRoB.
Results: We designed a structured pathway for assessing bias systematically, which helps classify a study into one of five categories for each RoB domain. A total of 34, 36, 26 and 20 scenarios were generated for sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment, respectively. Trial results showed that the iCRoB had a higher IRR across rater pairs than the original CRoB for every domain. The weighted κ was 0.71 and 0.81 for sequence generation respectively for CRoB and iCRoB; 0.53 and 0.61 for allocation concealment respectively for CRoB and iCRoB; 0.56 for blinding of participants and personnel in CRoB, 0.68 for blinding of participants and 0.70 for blinding of personnel in iCRoB; and 0.19 and 0.43 for blinding of outcome assessment respectively for CRoB and iCRoB.
Conclusions: We developed the iCRoB for making the judgement on RoB in reports of clinical trials. Our iCRoB showed a higher reliability than the current CRoB in all the domains examined. The iCRoB can be improved further by new contributions to the dictionary of scenarios and made easy by automating the bias assessment. Importantly, the iCRoB has created a possibility for automating the bias assessment.