Article type
Year
Abstract
Background: Evidence mapping is a systematic method for representing the evidence on a particular topic, with the resulting map facilitating identification of gaps in the literature. To date, there has been no evidence map of the methods used in overviews of systematic reviews, thus making it difficult to determine where there are gaps and hence what methods research should be undertaken as a priority.
Objectives:
1. To develop and populate a framework with methods that have, or may be used, in conducting, interpreting and reporting overviews (stage 1).
2. To create an evidence map of studies that have evaluated these methods (stage 2).
Methods: From a search of general and methods-specific databases, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for or used in overviews (stage 1). Studies evaluating the performance of these methods were identified from systematic reviews (SRs) and individual methods studies (stage 2). We described the evaluations and mapped them to the framework of methods developed in stage 1. This presentation considers initial, related steps (scope and purpose; eligibility criteria; search methods; data extraction), focusing on methods for which there are considerations unique to overviews.
Results: Forty-two studies identified methods relevant to one or more of the initial steps of conducting an overview; most with insufficient detail to operationalise methods. Synthesis across studies identified alternative approaches for each method (options). For example, in the data extraction step of the framework, options for handling discrepant data across SRs were to: 1) extract data from all reviews recording discrepancies; 2) extract data from one SR selected using prespecified criteria (e.g. most recent or highest quality); or 3) extract each data element (e.g. effect estimates, quality assessments) from the SR that meets decision rule criteria (e.g. most complete information on effect estimates; uses Cochrane 'Risk of bias' tool).
Conclusions: Our results provide a framework and inventory of studies evaluating the performance of methods for overviews.
Objectives:
1. To develop and populate a framework with methods that have, or may be used, in conducting, interpreting and reporting overviews (stage 1).
2. To create an evidence map of studies that have evaluated these methods (stage 2).
Methods: From a search of general and methods-specific databases, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for or used in overviews (stage 1). Studies evaluating the performance of these methods were identified from systematic reviews (SRs) and individual methods studies (stage 2). We described the evaluations and mapped them to the framework of methods developed in stage 1. This presentation considers initial, related steps (scope and purpose; eligibility criteria; search methods; data extraction), focusing on methods for which there are considerations unique to overviews.
Results: Forty-two studies identified methods relevant to one or more of the initial steps of conducting an overview; most with insufficient detail to operationalise methods. Synthesis across studies identified alternative approaches for each method (options). For example, in the data extraction step of the framework, options for handling discrepant data across SRs were to: 1) extract data from all reviews recording discrepancies; 2) extract data from one SR selected using prespecified criteria (e.g. most recent or highest quality); or 3) extract each data element (e.g. effect estimates, quality assessments) from the SR that meets decision rule criteria (e.g. most complete information on effect estimates; uses Cochrane 'Risk of bias' tool).
Conclusions: Our results provide a framework and inventory of studies evaluating the performance of methods for overviews.