Article type
Year
Abstract
Background: Overviews of reviews present unique methodological challenges for which authors need to plan. Through a systematic review, we have previously developed a comprehensive framework to catalogue method options available to overview authors. Here, we extend this work by linking the methods to commonly encountered scenarios in overviews.
Objectives: To provide guidance on methods for dealing with common scenarios in overviews.
Methods: We identified methodological challenges through systematic review of cross-sectional studies, guidance and commentaries on methods used in overviews. We used thematic synthesis to specify common scenarios for which authors need to plan, then mapped methods from our framework to each scenario.
Results: We included 42 studies for analysis. We identified seven common scenarios: overlapping data; discrepant data; missing or varying information; incomplete coverage of the overview question; out-of-date reviews; concerns about bias in included reviews; and discordant results or conclusions. We identified multiple methods for addressing each scenario. For example, a commonly cited approach for dealing with reviews with overlapping primary studies is to specify eligibility criteria (or decision rules) to select one review. However, multiple methods exist for addressing overlap at later steps of the overview. During synthesis, for example, authors can:
1) use decision rules to select one (or a subset) of meta-analyses with overlapping studies;
2) use statistical approaches to deal with overlap;
3) ignore overlap; or
4) acknowledge overlap as a limitation.
Alternatively, overlap may be addressed when assessing certainty of the evidence. Any of these approaches can be combined with methods to quantify and present overlap visually.
Conclusions: The methods literature often suggests a single approach for addressing common scenarios in overviews. However, our research identified multiple options that can be used at different steps, some of which can be combined. Our findings may usefully inform the methods to be used when planning an overview.
Patient or healthcare consumer involvement: None
Objectives: To provide guidance on methods for dealing with common scenarios in overviews.
Methods: We identified methodological challenges through systematic review of cross-sectional studies, guidance and commentaries on methods used in overviews. We used thematic synthesis to specify common scenarios for which authors need to plan, then mapped methods from our framework to each scenario.
Results: We included 42 studies for analysis. We identified seven common scenarios: overlapping data; discrepant data; missing or varying information; incomplete coverage of the overview question; out-of-date reviews; concerns about bias in included reviews; and discordant results or conclusions. We identified multiple methods for addressing each scenario. For example, a commonly cited approach for dealing with reviews with overlapping primary studies is to specify eligibility criteria (or decision rules) to select one review. However, multiple methods exist for addressing overlap at later steps of the overview. During synthesis, for example, authors can:
1) use decision rules to select one (or a subset) of meta-analyses with overlapping studies;
2) use statistical approaches to deal with overlap;
3) ignore overlap; or
4) acknowledge overlap as a limitation.
Alternatively, overlap may be addressed when assessing certainty of the evidence. Any of these approaches can be combined with methods to quantify and present overlap visually.
Conclusions: The methods literature often suggests a single approach for addressing common scenarios in overviews. However, our research identified multiple options that can be used at different steps, some of which can be combined. Our findings may usefully inform the methods to be used when planning an overview.
Patient or healthcare consumer involvement: None