Article type
Year
Abstract
Background: Since 1996, the Community Preventive Services Task Force (Task Force) has developed over 220 recommendations about public health interventions across a broad range of topics, based on systematic reviews, and compiled in The Community Guide. Integral to this process is assessing the generalizability of Task Force findings to a range of settings and populations encountered in public health, and options in implementing interventions, often despite incomplete empirical information.
Objective: To refine Community Guide methods to more systematically assess generalizability.
Methods Development: For each review, the Task Force considers three categories of potential effect modifiers (setting, population characteristics, and intervention characteristics), and identifies specific factors within these categories (e.g., ‘rural/urban/suburban’ under ‘setting’; ‘race/ethnicity’ under ‘population characteristics’; ‘tailoring’ under ‘implementation characteristics’) that are relevant to the intervention being reviewed. The Task Force draws conclusions on each potential effect modifier by considering: (1) an a priori hypothesis about the likely direction and magnitude of effect, developed by subject matter experts based on theoretical and experiential considerations for the intervention being studied (e.g., ‘likely effect modifier,’ ‘unlikely to be a substantial source of variability’); (2) the quality, quantity, and consistency of empirical evidence related to the factor of interest; and (3) the degree of concordance between the hypothesized relationships and the available evidence. The strength or tentativeness of Task Force conclusions regarding the generalizability of overall findings across settings, populations, and intervention characteristics vary based on these considerations. The two primary sources of evidence considered for assessing effect modification are between-study and within-study stratified analyses. Although the latter evidence has stronger internal validity (e.g., no collinearity problems), the former is much more commonly available in the public health literature, and is an important source of evidence on these vital issues.
Conclusion: These methods might assist others considering generalizability of findings from systematic reviews.
Objective: To refine Community Guide methods to more systematically assess generalizability.
Methods Development: For each review, the Task Force considers three categories of potential effect modifiers (setting, population characteristics, and intervention characteristics), and identifies specific factors within these categories (e.g., ‘rural/urban/suburban’ under ‘setting’; ‘race/ethnicity’ under ‘population characteristics’; ‘tailoring’ under ‘implementation characteristics’) that are relevant to the intervention being reviewed. The Task Force draws conclusions on each potential effect modifier by considering: (1) an a priori hypothesis about the likely direction and magnitude of effect, developed by subject matter experts based on theoretical and experiential considerations for the intervention being studied (e.g., ‘likely effect modifier,’ ‘unlikely to be a substantial source of variability’); (2) the quality, quantity, and consistency of empirical evidence related to the factor of interest; and (3) the degree of concordance between the hypothesized relationships and the available evidence. The strength or tentativeness of Task Force conclusions regarding the generalizability of overall findings across settings, populations, and intervention characteristics vary based on these considerations. The two primary sources of evidence considered for assessing effect modification are between-study and within-study stratified analyses. Although the latter evidence has stronger internal validity (e.g., no collinearity problems), the former is much more commonly available in the public health literature, and is an important source of evidence on these vital issues.
Conclusion: These methods might assist others considering generalizability of findings from systematic reviews.