Article type
Year
Abstract
Background: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system is considered to be a common, sensible and transparent approach to grading quality of evidence and strength of recommendations. Many guideline developers claimed that they have adopted GRADE system in their guidelines. The quality of evidence has four levels, high, moderate, low and very low.
Objectives: To investigate the agreement for grading the quality of evidence using GRADE system.
Methods: We randomly selected 26 Meta-analyses in CBM database which is similar with MEDLINE in China. And then we extracted 60 outcomes of them and assigned 2 groups randomly. Each group has 30 outcomes. We trained six raters and then assigned them to two groups randomly. Group 1 has two raters and group 2 has four raters. The Intraclass Correlation Coefficient (ICC) is used to measure the agreement.
Results: The ICC for two raters in Group 1 is 0.76 [95%CI: 0.5–0.89]. The ICC for two raters in Group 2 is 0.76 [95%CI: 0.5–0.89], the ICC for three raters in Group 2 is 0.84 [95%CI: 0.7–0.9], the ICC for four raters in Group 2 is 0.87 [95%CI: 0.78–0.94].
Conclusions: Based on the pilot research, in order to have better agreement for grading the quality of evidence using GRADE system, we recommended at least three raters for each outcome.
Objectives: To investigate the agreement for grading the quality of evidence using GRADE system.
Methods: We randomly selected 26 Meta-analyses in CBM database which is similar with MEDLINE in China. And then we extracted 60 outcomes of them and assigned 2 groups randomly. Each group has 30 outcomes. We trained six raters and then assigned them to two groups randomly. Group 1 has two raters and group 2 has four raters. The Intraclass Correlation Coefficient (ICC) is used to measure the agreement.
Results: The ICC for two raters in Group 1 is 0.76 [95%CI: 0.5–0.89]. The ICC for two raters in Group 2 is 0.76 [95%CI: 0.5–0.89], the ICC for three raters in Group 2 is 0.84 [95%CI: 0.7–0.9], the ICC for four raters in Group 2 is 0.87 [95%CI: 0.78–0.94].
Conclusions: Based on the pilot research, in order to have better agreement for grading the quality of evidence using GRADE system, we recommended at least three raters for each outcome.