Article type
Year
Abstract
Background: The Swedish National Institute of Public Health (SNIPH) received a government commission to determine which evidence grading system should be applied for assessing public health interventions in Sweden. The SNIPH subsequently adopted the GRADE framework.
Objectives: To describe the challenges of assessing the quality of evidence of public health interventions.
Methods: The SNIPH conducted in October 2011 an international workshop to discuss the challenges and potential solutions of different grading systems in public health. Also, the Institute organized in April 2012 a GRADE training workshop.
Results: We identified a number of challenges. (1) Public health interventions tend to be complex and multi-component. This has implications on defining the PICO question, conducting the search, interpreting results, assessing indirectness and inconsistency, and using process evaluation studies. (2) Outcomes of interest may include health outcomes at both the individual and community levels, and non-health outcomes such as environmental impact. Also, these outcomes might follow long causal pathways and have different levels of indirectness. An outcome framework helps in clarifying the relationship between different types of outcomes and in deciding which level provides ‘direct enough’ evidence. Such frameworks can be supported by both empirical and theoretical considerations. (3) Given settings will inevitably change across included studies, a higher level of heterogeneity may be acceptable when assessing inconsistency. (4) Assessment of public health interventions benefit from a wide range of study designs. In the GRADE approach, all observational studies starting as low quality evidence doesn’t help in discriminating between these different designs.
Conclusions: These challenges have either been addressed by, or are being discussed within the GRADE working group. SNIPH has received an additional government commission to investigate the use of GRADE to develop guidelines for public health interventions. The SNIPH will work with the GRADE working group on ways to address these challenges.
Objectives: To describe the challenges of assessing the quality of evidence of public health interventions.
Methods: The SNIPH conducted in October 2011 an international workshop to discuss the challenges and potential solutions of different grading systems in public health. Also, the Institute organized in April 2012 a GRADE training workshop.
Results: We identified a number of challenges. (1) Public health interventions tend to be complex and multi-component. This has implications on defining the PICO question, conducting the search, interpreting results, assessing indirectness and inconsistency, and using process evaluation studies. (2) Outcomes of interest may include health outcomes at both the individual and community levels, and non-health outcomes such as environmental impact. Also, these outcomes might follow long causal pathways and have different levels of indirectness. An outcome framework helps in clarifying the relationship between different types of outcomes and in deciding which level provides ‘direct enough’ evidence. Such frameworks can be supported by both empirical and theoretical considerations. (3) Given settings will inevitably change across included studies, a higher level of heterogeneity may be acceptable when assessing inconsistency. (4) Assessment of public health interventions benefit from a wide range of study designs. In the GRADE approach, all observational studies starting as low quality evidence doesn’t help in discriminating between these different designs.
Conclusions: These challenges have either been addressed by, or are being discussed within the GRADE working group. SNIPH has received an additional government commission to investigate the use of GRADE to develop guidelines for public health interventions. The SNIPH will work with the GRADE working group on ways to address these challenges.