Supplementary search methods versus bibliographic database searching to identify studies: early results from our Cochrane Methods Review

Article type
Authors
O'sullivan J1, Tomlinson D2, Nothnagel N2, Premji C3, Gonçalves Bradley M4, Freitag A5, Lopez Manzano S2, Benavente R2, Queder A6, Hafel U4, Court 7, Brown 7, Schauberger 8
1Population Health Sciences, Bristol Medical School (and then, i.e. two affiliations), University of Bristol, University Hospital Limerick, Raheen, Limerick, Ireland
2Population Health Sciences, Bristol Medical School, University of Bristol
3Libraries, University of Victoria, Victoria, British Columbia, Canada
4Symmetron Limited, London
5Department of General Pediatrics, Neonatology and Pediatric Cardiology, Medical Faculty, University Hospital Düsseldorf, Düsseldorf, Germany
6Department of Primary Care and Health Services Research, University Hospital Heidelberg, Heidelberg, Germany (and then, i.e. two affiliations) Medical Faculty Heidelberg, Department of Primary Care and Health Services Research, Heidelberg University, Heidelberg, Germany
7Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry , UK
8Independent Researcher, Glasgow
Abstract
Background: Bibliographic databases are the primary search method in Cochrane reviews, but there is evidence that supplementary (non-database) search methods may identify studies or study reports which might be missed by bibliographic database searching alone.

Objectives: This review aims to assess the effectiveness and resource requirements of supplementary search methods compared to bibliographic database searching in identifying studies and study reports.

Methods: We will search bibliographic databases and use supplementary search methods to identify studies. Two reviewers will independently screen reports for inclusion. Studies will be eligible for inclusion if they compared bibliographic database searching to any of the eligible supplementary search methods reported below (taken from the Cochrane Handbook), regardless of their study design:

1. Citation searching (including reference checking, and checking other reviews)
2. Contacting study authors and organisations
3. Handsearching
4. Regulatory agency sources and clinical study reports
5. Searching trials registry resources compared
6. Websearching (including search engines or searches of site-specific websites)

We will extract data from studies in duplicate, including study characteristics, the number of studies identified by supplementary searches and bibliographic databases, and resource use (time taken to perform search and/or costs involved). Two reviewers will independently assess risk of bias. We will summarise findings using narrative synthesis (or meta-analysis where feasible).

Effectiveness will be measured by i) number of unique studies and ii) number of unique study reports identified. Resource requirements will be measured by time taken and/or costs reported.

Results: We will present the protocol and early overview of included studies and preliminary review findings for this review.

Conclusions: The use of supplementary search methods may improve the identification of studies by providing access to reports missed or irretrievable by bibliographic database searching. The findings of this review will help to guide authors in the selection of search methods.

Relevance and Importance: Robust searches are paramount in ensuring that systematic reviews and evidence syntheses provide comprehensive and reliable evidence for decision-making. By evaluating the effectiveness and resource implications of supplementary search methods, this review aims to provide evidence to inform decision-making when selecting supplementary search methods in reviews of intervention effectiveness.