Article type
Year
Abstract
Background:
An essential step in systematic reviews is the assessment of the quality of included studies. The Academy of Nutrition and Dietetics recently updated their Quality Criteria Checklist (QCC) to assess risk of bias in randomized controlled trials. During this time, Cochrane also updated their 'Risk of bias' tool and developed the ROB 2.0.
Objective:
To compare the Academy of Nutrition and Dietetics updated QCC and Cochrane's ROB 2.0 as risk of bias assessment tools for randomized controlled trials within a systematic review.
Methods:
We included 10 RCTs in this comparison study. Two review authors independently reviewed the 10 articles using the updated QCC and ROB 2.0 tool. The updated QCC contains six validity questions (with signalling questions for each main question) focusing on selection bias, detection bias, performance bias, attrition bias and reporting bias. The ROB 2.0 contains five validity questions (with signalling questions for each main question). We calculated inter-observer agreement for each risk of bias domain for each tool. First, authors mapped the individual questions for both the tools to Cochrane's risk of bias domains (selection, attrition, detection, performance, other bias). Next, we analyzed comparisons between the ratings for the two tools for each domain.
Results:
Mapping of each question for the two tools indicated that both the updated QCC and ROB 2.0 questions were very well distributed across all the risk of bias domains. Results indicated that inter-observer agreement was higher for QCC compared to ROB 2.0 for all risk of bias domains. Performance bias and detection bias domains had the lowest agreement (performance bias: 60% for QCC, 40% for ROB 2.0; detection bias: 50% for QCC, 40% for ROB 2.0). We are still in the process of calculating overall inter-observer reliability status for both the tools and investigating which signalling questions contribute to more discrepancy/disagreement between observers.
Conclusion:
The updated QCC has higher inter-observer agreement compared to ROB 2.0. However, both tools are easy to use and time-efficient.
Patient or healthcare consumer involvement:
Not involved in the first phase of comparing tools; however, a patient advocate is involved in each guideline project.
An essential step in systematic reviews is the assessment of the quality of included studies. The Academy of Nutrition and Dietetics recently updated their Quality Criteria Checklist (QCC) to assess risk of bias in randomized controlled trials. During this time, Cochrane also updated their 'Risk of bias' tool and developed the ROB 2.0.
Objective:
To compare the Academy of Nutrition and Dietetics updated QCC and Cochrane's ROB 2.0 as risk of bias assessment tools for randomized controlled trials within a systematic review.
Methods:
We included 10 RCTs in this comparison study. Two review authors independently reviewed the 10 articles using the updated QCC and ROB 2.0 tool. The updated QCC contains six validity questions (with signalling questions for each main question) focusing on selection bias, detection bias, performance bias, attrition bias and reporting bias. The ROB 2.0 contains five validity questions (with signalling questions for each main question). We calculated inter-observer agreement for each risk of bias domain for each tool. First, authors mapped the individual questions for both the tools to Cochrane's risk of bias domains (selection, attrition, detection, performance, other bias). Next, we analyzed comparisons between the ratings for the two tools for each domain.
Results:
Mapping of each question for the two tools indicated that both the updated QCC and ROB 2.0 questions were very well distributed across all the risk of bias domains. Results indicated that inter-observer agreement was higher for QCC compared to ROB 2.0 for all risk of bias domains. Performance bias and detection bias domains had the lowest agreement (performance bias: 60% for QCC, 40% for ROB 2.0; detection bias: 50% for QCC, 40% for ROB 2.0). We are still in the process of calculating overall inter-observer reliability status for both the tools and investigating which signalling questions contribute to more discrepancy/disagreement between observers.
Conclusion:
The updated QCC has higher inter-observer agreement compared to ROB 2.0. However, both tools are easy to use and time-efficient.
Patient or healthcare consumer involvement:
Not involved in the first phase of comparing tools; however, a patient advocate is involved in each guideline project.