Article type
Year
Abstract
Background: Search strategies for systematic reviews have to manage a trade-off between sensitivity and precision. Search filters are useful for conducting searches in Cochrane Reviews and the Cochrane RCT filter is commonly used. The increasing urgency for the retrieval of studies on the health impact of climate change (across all areas of health and levels of health system) requires a search strategy which maximises sensitivity as well as precision. Previous searches have attempted to retrieve the totality of research in this area (as required for any review), but none have reported on the sensitivity or precision of their search strategies.
Objectives: We aimed to create search filters in the area of climate change and health in Medline and Embase and to improve on the methodology of creating search filters.
Methods: The climate change and health terms were obtained using objective methods (such as online word frequency software (www.writewords.org)) and subjective methods (using terms from previously employed strategies and terms already known to the searchers). These terms, using systematically randomised samples, were then subject to univariate analysis, to establish relevance by counting relevant articles in the samples, and multivariate analysis, to explore the relationship between terms when searched for in the same search string.
Once the strategies' search terms were finalised the full results were screened for inclusion in Endnote. Precision was established by dividing the number of relevant articles by the total number of articles retrieved. An estimate of sensitivity was based on running samples (N=1000) for each database using only the climate change terms. Articles relevant to health were counted and, by working out the percentage captured by our search filter, an estimate of sensitivity was established. We aimed for an acceptable precision of ≥ 50% and an estimate of sensitivity ≥ 90%.
Results: Following deduplication, 7236 articles were retrieved from Medline, of which 4924 were counted as true positives (68.04% precision). 10602 articles were retrieved from Embase, of which 6377 were counted as true positives (60.14% precision). The samples used for estimating sensitivity found 155 articles out of 1000 relevant to health for Medline, of which 145 were identified by our search (93.5% estimated sensitivity). In Embase this number is 164 out of 1000, of which 15 were identified by our search (90.9% estimated sensitivity).
Conclusions: The performance measures in these search strategies, for both Medline and Embase, are both higher than those generally accepted for systematic reviews. The methods we used to create these search strategies may be generalizable to other filters useful for Cochrane Reviews.
Patient or healthcare consumer involvement: Due to the nature of this study, patients or healthcare consumers were not involved.
Objectives: We aimed to create search filters in the area of climate change and health in Medline and Embase and to improve on the methodology of creating search filters.
Methods: The climate change and health terms were obtained using objective methods (such as online word frequency software (www.writewords.org)) and subjective methods (using terms from previously employed strategies and terms already known to the searchers). These terms, using systematically randomised samples, were then subject to univariate analysis, to establish relevance by counting relevant articles in the samples, and multivariate analysis, to explore the relationship between terms when searched for in the same search string.
Once the strategies' search terms were finalised the full results were screened for inclusion in Endnote. Precision was established by dividing the number of relevant articles by the total number of articles retrieved. An estimate of sensitivity was based on running samples (N=1000) for each database using only the climate change terms. Articles relevant to health were counted and, by working out the percentage captured by our search filter, an estimate of sensitivity was established. We aimed for an acceptable precision of ≥ 50% and an estimate of sensitivity ≥ 90%.
Results: Following deduplication, 7236 articles were retrieved from Medline, of which 4924 were counted as true positives (68.04% precision). 10602 articles were retrieved from Embase, of which 6377 were counted as true positives (60.14% precision). The samples used for estimating sensitivity found 155 articles out of 1000 relevant to health for Medline, of which 145 were identified by our search (93.5% estimated sensitivity). In Embase this number is 164 out of 1000, of which 15 were identified by our search (90.9% estimated sensitivity).
Conclusions: The performance measures in these search strategies, for both Medline and Embase, are both higher than those generally accepted for systematic reviews. The methods we used to create these search strategies may be generalizable to other filters useful for Cochrane Reviews.
Patient or healthcare consumer involvement: Due to the nature of this study, patients or healthcare consumers were not involved.