Article type
Year
Abstract
Background: Oregon Health & Science University’s Center for Evidence-based Policy (Center) conducts evidence summaries for policy makers using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess overall quality of evidence (QoE). Many policymakers find it difficult to interpret and apply GRADE QoE definitions.
Objectives: Assess policymakers’ familiarity with GRADE and preference for five different QoE definitions.
Methods: Policymakers participating in Center projects were invited to complete a 24-item web-based survey assessing familiarity and preference of five QoE definitions including: GRADE, Agency for Healthcare Research and Quality (AHRQ), Evidence-based Practice Center (EPC), and three alternative definitions developed by the Center. Familiarity with GRADE was based on a 2-item composite variable. Policymaker degree of certainty that findings will remain stable over time was rated from 0% to 100% for each QoE definition. Odds ratios were used to assess the association between familiarity and definition preference.
Results: Out of 92 policymakers, 42 (46%) completed the survey with 25 (60%) indicating that they were familiar with GRADE. Nearly half of respondents preferred the EPC definitions. Fifteen (38%) preferred alternative definitions, but none selected a brief lay terminology definition. Only five (13%) preferred the standard GRADE definition. For each GRADE QoE level, the median per cent certainty that findings from evidence would remain stable was 90% for high QoE, 75% for moderate QoE, and 45% for low QoE. Although 29 (69%) policymakers had experience with GRADE, none were extremely familiar with it. Familiarity with GRADE was not associated with a preference for standard (GRADE or EPC) QoE definitions compared to alternatives (OR = 1.3, 95% CI 0.4 to 5.1; P value 0.7).
Conclusions: The EPC definitions were preferred by policymakers. Despite having experience with GRADE, no policymaker felt extremely familiar with it. Policymakers’ familiarity was not associated with definition preference. Strategies to enhance policymakers’ understanding and application of GRADE ratings are needed.
Objectives: Assess policymakers’ familiarity with GRADE and preference for five different QoE definitions.
Methods: Policymakers participating in Center projects were invited to complete a 24-item web-based survey assessing familiarity and preference of five QoE definitions including: GRADE, Agency for Healthcare Research and Quality (AHRQ), Evidence-based Practice Center (EPC), and three alternative definitions developed by the Center. Familiarity with GRADE was based on a 2-item composite variable. Policymaker degree of certainty that findings will remain stable over time was rated from 0% to 100% for each QoE definition. Odds ratios were used to assess the association between familiarity and definition preference.
Results: Out of 92 policymakers, 42 (46%) completed the survey with 25 (60%) indicating that they were familiar with GRADE. Nearly half of respondents preferred the EPC definitions. Fifteen (38%) preferred alternative definitions, but none selected a brief lay terminology definition. Only five (13%) preferred the standard GRADE definition. For each GRADE QoE level, the median per cent certainty that findings from evidence would remain stable was 90% for high QoE, 75% for moderate QoE, and 45% for low QoE. Although 29 (69%) policymakers had experience with GRADE, none were extremely familiar with it. Familiarity with GRADE was not associated with a preference for standard (GRADE or EPC) QoE definitions compared to alternatives (OR = 1.3, 95% CI 0.4 to 5.1; P value 0.7).
Conclusions: The EPC definitions were preferred by policymakers. Despite having experience with GRADE, no policymaker felt extremely familiar with it. Policymakers’ familiarity was not associated with definition preference. Strategies to enhance policymakers’ understanding and application of GRADE ratings are needed.