From summary to synthesis: a review of statistical synthesis and presentation methods used in complex reviews

Article type
Authors
McKenzie JE1, Brennan S1, Page MJ1, Chau M2, Kramer S1, Bosch M2
1School of Public Health and Preventive Medicine, Monash University, Australia
2Central Clinical School, Monash University, Australia
Abstract
Background: Systematic reviews evaluating policy, public health, and health service delivery interventions involve additional complexity compared with clinical reviews. This complexity arises from multifaceted interventions being evaluated across a wide range of settings, conditions, outcomes, and study designs. Application ofmeta-analytical methods in these reviews can be challenging, and the use of ad hoc approaches (e.g. counting the number of studies with statistically significant results) may under-utilise or misrepresent available research. To date, there has been no evaluation of the different synthesis and presentation methods being employed in complex systematic reviews.

Objective: 1. To describe and estimate the prevalence of different synthesis and presentation methods used in a sample of complex systematic reviews. 2. To describe the advantages and disadvantages of each of the identified synthesis and presentation methods.

Methods: Systematic reviews published between 2008 and 2012 were identified from the Health Systems Evidence Database and the Effective Practice and Organisation of Care Group. The resulting sampling frame was stratified by two types of reviews, those produced within, and externally to, The Cochrane Collaboration. A sample of 50 reviews from each stratum was randomly selected. Data extracted included: diversity of interventions, settings, conditions, outcomes, and study designs; use of outcome categories; synthesis and presentation methods; and rationale for the choice of methods. Results will be summarised using descriptive statistics. The advantages and disadvantages of each of the identified methods will be sought from methods literature.

Results: Data extraction is ongoing. Results will be available at the Colloquium.

Conclusion: Systematic reviews used to inform policy and public health decision making have broad population impact. Synthesis and presentation methods are needed that offer effective and fair presentation of results, and make best use of available research. This research will describe and quantify the use of different methods, identifying where guidance is needed.