Checking reference lists to find additional studies for systematic reviews

Article type
Authors
Horsley T, Dingwall O, Tetzlaff J, Sampson M
Abstract
Background: In order to identify all potential sources of evidence for systematic reviews, authors may employ a number of strategies including the development of highly sensitive search strategies, handsearching, and contacting authors. Another commonly applied methodology is the process of checking reference lists (CRL) of included studies. This methodology is currently recommended by systematic review manuals and is commonly used by review authors. Evidence-based practice would weigh the sensitivity and specificity of such an approach, against the resource implications, and the evidence regarding the potential to influence overall estimates/outcomes of the review. Objectives: To systematically review evidence examining the methodology of checking reference lists to identify potentially relevant resources for inclusion in systematic reviews. Methods: All studies were considered to be eligible for inclusion if they examined the methodology of CRL (distinct from other search methodologies) for identifying potentially relevant evidence for inclusion within systematic reviews. Additional yield (unique yield specifies that the record or source was not found through any other search methodologies) of the methodology of CRL for identifying potentially relevant resources for inclusion within systematic reviews. Results: We identified 3989 records for initial screening by title and abstract. The full-text reports of 155 records were then retrieved and assessed in duplicate by two (of four) authors for relevance. In total, 12 studies met all inclusion criteria and form the basis for our analysis. We did not identify any studies with a contemporaneous comparator (RCT, CCT, ITS, CBA) and were unable to estimate risk of bias formally. Conclusions: Evidence suggests there is utility in using CRL to supplement poorly developed search strategies that miss relevant indexed citations and detect document type not usually indexed within bibliographic databases. Future research, using prospective designs is warranted.