Background: Systematic reviews have been accepted as providing a transparent and consistent way of obtaining research evidence on effectiveness of interventions in a way that minimizes bias. Meta-analysis, which combines quantitative information from multiple studies, is considered the gold standard in the hierarchy of evidence for intervention effectiveness. However, widespread application of quantitative synthesis methods to public health is hampered by a number of well documented methodological challenges.

Objectives: To review the methods currently used to synthesise evidence in public health evaluations and to demonstrate the availability of more sophisticated approaches.

Methods: A systematic review of NICE public health appraisals published between 2006 and 2012 was performed to assess the methods used for the synthesis of effectiveness evidence. The ability of new developments in evidence synthesis methodology to address the challenges and opportunities present in a public health context is demonstrated.

Results: Only 7 (18%) of the 39 NICE appraisals included in the review performed pairwise meta-analyses as part of the effectiveness review with one of these also including a network meta-analysis. Of the remainder, 31 (79.5%) consisted of narrative summaries of the evidence only, and 1 (2.5%) had no review of evidence. Heterogeneity of outcomes and interventions were the main reasons given for not pooling the data. Exploration of quantitative syntheses methods shows that pairwise meta-analyses can be extended to incorporate individual participant data (where it is available), extend the number of interventions being compared by using a network meta-analysis, and adjust for subject-level and summary covariates. All these can contribute to ensuring the analysis answers directly the policy relevant questions.

Conclusions: More sophisticated methods in evidence synthesis should be considered to make evaluations in public health more vigorous.