Article type
Year
Abstract
Background: Many organizations have adopted the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach to rate researcher confidence in an available body of evidence. GRADE’s definition of quality of evidence (QoE) links individual grades to the degree of confidence that estimates are close to the true effect (and thus will remain stable as new evidence accrues).
Objectives: To determine the predictive validity of the GRADE approach (i.e. whether GRADE discriminates reliably between evidence that remains stable and evidence that changes as new studies emerge).
Research Design/Methods: To determine the predictive validity, we randomly assigned 13 researchers who are producers and users of systematic reviews using the GRADE approach from six US Evidence-based Practice Centers and Cochrane Austria 160 bodies of evidence to grade. Using likelihoods from the survey as reference points, we calculated c-statistics to determine the predictive validity.
Results: GRADE did not discriminate well between bodies of evidence that remained stable and those that changed (c-scores 0.56 to 0.58).
Conclusions: GRADE is a suitable method for systematic review producers to convey uncertainties to users. The predictive validity of GRADE was compromised by grades of QoE that seemed, in general, too low.
Objectives: To determine the predictive validity of the GRADE approach (i.e. whether GRADE discriminates reliably between evidence that remains stable and evidence that changes as new studies emerge).
Research Design/Methods: To determine the predictive validity, we randomly assigned 13 researchers who are producers and users of systematic reviews using the GRADE approach from six US Evidence-based Practice Centers and Cochrane Austria 160 bodies of evidence to grade. Using likelihoods from the survey as reference points, we calculated c-statistics to determine the predictive validity.
Results: GRADE did not discriminate well between bodies of evidence that remained stable and those that changed (c-scores 0.56 to 0.58).
Conclusions: GRADE is a suitable method for systematic review producers to convey uncertainties to users. The predictive validity of GRADE was compromised by grades of QoE that seemed, in general, too low.