Article type
Year
Abstract
Background:
The synthesis component of systematic reviews is often narrowly considered to include only statistical methods, primarily meta-analysis. However, synthesis is a process, beginning with (i) defining the groupings of populations, interventions and outcome to be compared within the review (the ‘PICO for synthesis’), (ii) examining the characteristics of the available studies, and (iii) applying appropriate synthesis methods from among multiple options. This study examines two intertwined aspects of synthesis ((i) and (iii)) that commonly challenge authors and end users of systematic reviews. First, the ‘PICO for synthesis’ involves decisions about which studies and outcome data will be combined in each analysis - decisions that affect the review’s findings. Second, meta-analysis is not used in around a third of systematic reviews, requiring other methods of summary and synthesis. Where the PICO for synthesis is not clearly defined, or synthesis methods are not applied optimally, this can reduce transparency, replicability and end users’ ability to interpret the review’s findings. Guidance on both aspects is included in the 2019 Cochrane Handbook, but further work is required to explore the feasibility and impact of applying the guidance in practice.
Objectives:
To identify and describe current practice in systematic reviews in relation to structuring the PICO for synthesis and methods for synthesis when meta-analysis is not used.
Methods:
We randomly sampled systematic reviews of public health and health systems interventions indexed in 2018 in the Health Evidence and Health Systems Evidence databases. Eligible reviews were systematic reviews of primary studies, assessing the quantitative effects of health interventions, including at least two included studies, and published in English. We aimed to select a sample size of at least 100, including reviews both with and without meta-analysis. Two authors independently screened studies for inclusion. One author extracted data on the eligibility criteria (PICO for the review); approaches to grouping populations, interventions and outcomes for synthesis (PICO for synthesis); and the summary and synthesis methods used (e.g. tabulation, visual displays, text-based description and statistical synthesis methods such as combining P values, vote counting based on direction of effect and meta-analysis). A second author undertook independent data extraction for a subsample of reviews. Descriptive statistics were used to summarise the findings.
Results:
865 unique records published in 2018 were retrieved from the databases. 166 reviews were randomly sampled and the title and abstract screened. 151 reviews were screened in full text. At the time of submission, full results were not available, but will be presented at the Colloquium.
Conclusions:
This study presents a detailed description of current practice in specifying PICO for synthesis and the use of summary and synthesis methods other than meta-analysis.
Patient or healthcare consumer involvement: None in this study.
The synthesis component of systematic reviews is often narrowly considered to include only statistical methods, primarily meta-analysis. However, synthesis is a process, beginning with (i) defining the groupings of populations, interventions and outcome to be compared within the review (the ‘PICO for synthesis’), (ii) examining the characteristics of the available studies, and (iii) applying appropriate synthesis methods from among multiple options. This study examines two intertwined aspects of synthesis ((i) and (iii)) that commonly challenge authors and end users of systematic reviews. First, the ‘PICO for synthesis’ involves decisions about which studies and outcome data will be combined in each analysis - decisions that affect the review’s findings. Second, meta-analysis is not used in around a third of systematic reviews, requiring other methods of summary and synthesis. Where the PICO for synthesis is not clearly defined, or synthesis methods are not applied optimally, this can reduce transparency, replicability and end users’ ability to interpret the review’s findings. Guidance on both aspects is included in the 2019 Cochrane Handbook, but further work is required to explore the feasibility and impact of applying the guidance in practice.
Objectives:
To identify and describe current practice in systematic reviews in relation to structuring the PICO for synthesis and methods for synthesis when meta-analysis is not used.
Methods:
We randomly sampled systematic reviews of public health and health systems interventions indexed in 2018 in the Health Evidence and Health Systems Evidence databases. Eligible reviews were systematic reviews of primary studies, assessing the quantitative effects of health interventions, including at least two included studies, and published in English. We aimed to select a sample size of at least 100, including reviews both with and without meta-analysis. Two authors independently screened studies for inclusion. One author extracted data on the eligibility criteria (PICO for the review); approaches to grouping populations, interventions and outcomes for synthesis (PICO for synthesis); and the summary and synthesis methods used (e.g. tabulation, visual displays, text-based description and statistical synthesis methods such as combining P values, vote counting based on direction of effect and meta-analysis). A second author undertook independent data extraction for a subsample of reviews. Descriptive statistics were used to summarise the findings.
Results:
865 unique records published in 2018 were retrieved from the databases. 166 reviews were randomly sampled and the title and abstract screened. 151 reviews were screened in full text. At the time of submission, full results were not available, but will be presented at the Colloquium.
Conclusions:
This study presents a detailed description of current practice in specifying PICO for synthesis and the use of summary and synthesis methods other than meta-analysis.
Patient or healthcare consumer involvement: None in this study.