Article type
Year
Abstract
Background:
Systematic reviews evaluating public health and health services interventions involve additional complexity compared with clinical reviews. This complexity arises from multi-faceted interventions, evaluated in varied settings, conditions, outcomes, and study designs. Application of meta-analytical methods in these reviews can be challenging, and the use of ad hoc approaches (e.g. counting the number of studies with statistically significant results) may under-utilize or misrepresent available research. To date, there has been no evaluation of the different synthesis and presentation methods used in complex reviews.
Objectives:
1. To describe and estimate the prevalence of different synthesis and presentation methods used in a sample of complex systematic reviews.
2. To describe the advantages and disadvantages of each of the identified synthesis and presentation methods.
Methods:
Systematic reviews published between 2008 and 2012 were identified from the Health Systems Evidence Database. A sample of 50 Cochrane Reviews was randomly selected. Data extracted included: diversity of interventions, settings, conditions, outcomes, and study designs; use of outcome categories; synthesis and presentation methods; and rationale for the choice of methods. Results were summarized using descriptive statistics.
Results:
Outcome categories (domains) were used in 78% of reviews to summarize the results of included studies, but were less commonly used to aid synthesis (30% of reviews). Synthesis was undertaken in 60% of reviews, most commonly meta-analysis (Figure 1). Forest plots were used in 60% of reviews but rarely in reviews with no syntheses (5%) compared with reviews with synthesis (97%); risk difference -92% (95%CI -100% to -80%). Only 30% of reviews pre-specified selection methods for dealing with multiplicity of outcomes. Synthesis was more common in these reviews (87%) compared with reviews where no selection method was provided (49%); risk difference 38% (95%CI 14% to 62%).
Conclusions:
Outcome categorization and selection methods can facilitate presentation and synthesis in complex reviews, enabling best use of available research.
Systematic reviews evaluating public health and health services interventions involve additional complexity compared with clinical reviews. This complexity arises from multi-faceted interventions, evaluated in varied settings, conditions, outcomes, and study designs. Application of meta-analytical methods in these reviews can be challenging, and the use of ad hoc approaches (e.g. counting the number of studies with statistically significant results) may under-utilize or misrepresent available research. To date, there has been no evaluation of the different synthesis and presentation methods used in complex reviews.
Objectives:
1. To describe and estimate the prevalence of different synthesis and presentation methods used in a sample of complex systematic reviews.
2. To describe the advantages and disadvantages of each of the identified synthesis and presentation methods.
Methods:
Systematic reviews published between 2008 and 2012 were identified from the Health Systems Evidence Database. A sample of 50 Cochrane Reviews was randomly selected. Data extracted included: diversity of interventions, settings, conditions, outcomes, and study designs; use of outcome categories; synthesis and presentation methods; and rationale for the choice of methods. Results were summarized using descriptive statistics.
Results:
Outcome categories (domains) were used in 78% of reviews to summarize the results of included studies, but were less commonly used to aid synthesis (30% of reviews). Synthesis was undertaken in 60% of reviews, most commonly meta-analysis (Figure 1). Forest plots were used in 60% of reviews but rarely in reviews with no syntheses (5%) compared with reviews with synthesis (97%); risk difference -92% (95%CI -100% to -80%). Only 30% of reviews pre-specified selection methods for dealing with multiplicity of outcomes. Synthesis was more common in these reviews (87%) compared with reviews where no selection method was provided (49%); risk difference 38% (95%CI 14% to 62%).
Conclusions:
Outcome categorization and selection methods can facilitate presentation and synthesis in complex reviews, enabling best use of available research.