Literature searching for systematic reviews and health technology assessments (HTAs): can one rule fit all?

Tags: Oral
J B, JM G, CJ L

Objectives: To provide a critique, and discussion of the implications for practice, of selected recent research which investigates the trade-off between the resources (in particular time and cost) allocated to literature searching and the quality of systematic reviews and HTAs [1,2,3,4].

Methods: A survey and critical appraisal of recent research was undertaken. The methodological robustness and the generalisability of the results and conclusions, including the implications for practice, for future systematic reviews and HTAs, were considered.

Results: Egger et al. and Royle & Waugh provide an interesting analysis of some aspects of literature searching practice for systematic reviews and for HTAs[1,2]. Neither study, however, assessed the quality of the search strategies used to identify eligible studies for inclusion in systematic reviews or HTAs. Consequently, it is our view that the strength of their recommendations is not fully supported by their methods, or when seen in the context of other relevant research.

Conclusion: This critique will argue that the decision about where to search for eligible studies to include in any particular systematic review or HTA is complex, and that attempts to devise a "one-rule-fits-all" approach are unlikely to succeed in the near future. The question about how to search effectively, for example, issues around search strategy design, remains unanswered in these studies. The decision about which sources to search should be based on evidence from existing reviews and scoping searches and advice from experts in the field and from information specialists with experience of information retrieval for systematic reviews and HTAs.

References: 1. Egger M, Juni P, Bartlett C, Holenstein F and Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1-76. 2. Royle P and Waugh N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess. 2003; 7(34). 3. Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for results of clinical trials (Cochrane Methodology Review). In: The Cochrane Library, Issue 1, 2004. Chichester, UK: John Wiley & Sons, Ltd. 4. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions (Cochrane Methodology Review). In: The Cochrane Library, Issue 1, 2004. Chichester, UK: John Wiley & Sons, Ltd.