Article type
Year
Abstract
Background: The capacity of systematic reviewers to present global summaries of evidence is expanding, and public health decision-makers working locally are increasingly presented with the challenge of how to interpret global review evidence and assess its meaning in local contexts.
Objectives:
1. Establish how systematic reviewers of public health interventions assess the generalisability of the evidence that they encounter and produce
2. Present methods for undertaking analyses of existing secondary data sources to assess and enhance the generalisability of review evidence.
Methods: We reviewed some of the main tools used by systematic reviewers working in public health to assess the generalisability of evidence and found that current practice is limited. We will present a framework of how the epistemological foci of secondary (observational) data differ, but identify complementary ways in which the further analysis of existing secondary data can aid in the interpretation of meta-analyses.
Results: We identify three main approaches. The first approach involves purposeful exploration before starting a review to ensure that the findings are relevant to an inference population; the second involves purposeful exploration after a review has been conducted, where we present a framework and examples of potential avenues of enquiry; the final approach involves recalibration of the results to weight studies differentially based on their similarity to conditions in an inference population.
Conclusions: Generalisability as a concept has historically been deprioritised in trial literature and it has become standard practice for meta-analysts to synthesise evidence from conceptually discordant settings and populations. Analysis of existing surveys and routine datasets represents an important, but overlooked, vehicle for achieving a more nuanced understanding and treatment of context, necessary for decision-making.
Objectives:
1. Establish how systematic reviewers of public health interventions assess the generalisability of the evidence that they encounter and produce
2. Present methods for undertaking analyses of existing secondary data sources to assess and enhance the generalisability of review evidence.
Methods: We reviewed some of the main tools used by systematic reviewers working in public health to assess the generalisability of evidence and found that current practice is limited. We will present a framework of how the epistemological foci of secondary (observational) data differ, but identify complementary ways in which the further analysis of existing secondary data can aid in the interpretation of meta-analyses.
Results: We identify three main approaches. The first approach involves purposeful exploration before starting a review to ensure that the findings are relevant to an inference population; the second involves purposeful exploration after a review has been conducted, where we present a framework and examples of potential avenues of enquiry; the final approach involves recalibration of the results to weight studies differentially based on their similarity to conditions in an inference population.
Conclusions: Generalisability as a concept has historically been deprioritised in trial literature and it has become standard practice for meta-analysts to synthesise evidence from conceptually discordant settings and populations. Analysis of existing surveys and routine datasets represents an important, but overlooked, vehicle for achieving a more nuanced understanding and treatment of context, necessary for decision-making.