Background:

Systematic reviews evaluating public health and health services interventions involve additional complexity compared with clinical reviews. This complexity arises from multi-faceted interventions, evaluated in varied settings, conditions, outcomes, and study designs. Application of meta-analytical methods in these reviews can be challenging, and the use of ad hoc approaches (e.g. counting the number of studies with statistically significant results) may under-utilize or misrepresent available research. To date, there has been no evaluation of the different synthesis and presentation methods used in complex reviews.

Objectives:

1. To describe and estimate the prevalence of different synthesis and presentation methods used in a sample of complex systematic reviews.
2. To describe the advantages and disadvantages of each of the identified synthesis and presentation methods.

Methods:

Systematic reviews published between 2008 and 2012 were identified from the Health Systems Evidence Database. A sample of 50 Cochrane Reviews was randomly selected. Data extracted included: diversity of interventions, settings, conditions, outcomes, and study designs; use of outcome categories; synthesis and presentation methods; and rationale for the choice of methods. Results were summarized using descriptive statistics.

Results:

Outcome categories (domains) were used in 78% of reviews to summarize the results of included studies, but were less commonly used to aid synthesis (30% of reviews). Synthesis was undertaken in 60% of reviews, most commonly meta-analysis (Figure 1). Forest plots were used in 60% of reviews but rarely in reviews with no syntheses (5%) compared with reviews with synthesis (97%); risk difference -92% (95%CI -100% to -80%). Only 30% of reviews pre-specified selection methods for dealing with multiplicity of outcomes. Synthesis was more common in these reviews (87%) compared with reviews where no selection method was provided (49%); risk difference 38% (95%CI 14% to 62%).

Conclusions:

Outcome categorization and selection methods can facilitate presentation and synthesis in complex reviews, enabling best use of available research.