Article type
Year
Abstract
Background: An effective education program had been developed in Korea since 2013 to train qualified appraisers of guidelines. To alleviate discrepancies among appraisers (multidisciplinary physicians), a scoring guide that consists of 92 guides for anchor points 1, 3, 5, and 7 in 23 items of AGREE II instrument has been developed and applied by the Executive Committee for clinical practice guidelines (CPGs), the Korean Academy of Medical Sciences in Korea.
Objectives: To train and examine AGREE II scoring guide for time-saving and increasing reliability for the assessment of quality of CPGs among appraisers.
Methods: Three hours of an education program was provided to 29 physicians in 2013 and 20 physicians in 2014. There were three different training subjects: understanding the AGREE II instrument and a scoring guide; learning how to apply the scoring guide; and how to evaluate each of the 23 items and assign a score. The guideline for prevention and treatment of metabolic syndrome in primary care and the guideline for the diagnosis and treatment of Helicobacter pylori infection in Korea, 2013 Revised Edition, were assessed using a Korean AGREE II scoring guide in 2013 and 2014. Each item was rated on a 7-point scale by participants. Disagreement was defined by more than four differences in score among appraisers.
Results: Item 5 in domain 2 (stakeholder involvement) showed the highest disagreement (27.6%, 8 of 29), and five items (items 1, 4, 6, 7 and 15) showed no disagreement in 2013. Items18 and 19 in domain 5 (applicability) showed high disagreement (35% and 40.0%), and eight items (item 1, 2, 6, 7, 8, 10, 12 and 17) showed no disagreement in 2014.
Conclusions: We demonstrated that the Korean AGREE II scoring guide was a useful tool to train appraisers of CPGs; it provides a capable time-saving method to teach how to rate with a low disagreement rate.
Objectives: To train and examine AGREE II scoring guide for time-saving and increasing reliability for the assessment of quality of CPGs among appraisers.
Methods: Three hours of an education program was provided to 29 physicians in 2013 and 20 physicians in 2014. There were three different training subjects: understanding the AGREE II instrument and a scoring guide; learning how to apply the scoring guide; and how to evaluate each of the 23 items and assign a score. The guideline for prevention and treatment of metabolic syndrome in primary care and the guideline for the diagnosis and treatment of Helicobacter pylori infection in Korea, 2013 Revised Edition, were assessed using a Korean AGREE II scoring guide in 2013 and 2014. Each item was rated on a 7-point scale by participants. Disagreement was defined by more than four differences in score among appraisers.
Results: Item 5 in domain 2 (stakeholder involvement) showed the highest disagreement (27.6%, 8 of 29), and five items (items 1, 4, 6, 7 and 15) showed no disagreement in 2013. Items18 and 19 in domain 5 (applicability) showed high disagreement (35% and 40.0%), and eight items (item 1, 2, 6, 7, 8, 10, 12 and 17) showed no disagreement in 2014.
Conclusions: We demonstrated that the Korean AGREE II scoring guide was a useful tool to train appraisers of CPGs; it provides a capable time-saving method to teach how to rate with a low disagreement rate.