A comparison of handsearching versus MEDLINE searching to identify reports of randomized controlled trials.

Hopewell S, Clarke M, Lusher A, Westby M, Lefebvre C

Background: To compare handsearching versus MEDLINE searching for the identification of reports of randomized controlled trials (RCTs) in specialized health care journals.

Methods: 22 specialised health care journals, published in the UK, were handsearched from 1970 to 1998, as part of a European Union BIOMED programme funded project (BMH4-CT98-3803), for all reports of RCTs (as defined by the Cochrane Collaboration for inclusion in The Cochrane Controlled Trials Register). The reports of RCTs from three years per journal were selected and form one element of this study. A MEDLINE search using the Publication Type terms RANDOMIZED CONTROLLED TRIAL and CONTROLLED CLINICAL TRIAL was also performed for the same journal years. The reports of trials retrieved by handsearching were then compared against those retrieved from the MEDLINE search, to identify differences in retrieval between the two techniques. Reports of RCTs identified by MEDLINE but not found by handsearching, were individually assessed to see if they met the Cochrane eligibility criteria as a report of an RCT.

Results: A total of 715 reports of RCTs were found by using a combination of both handsearching and MEDLINE searching. Of these, 369 (52%) were identified only by handsearching and 32 (4%) were identified only by MEDLINE. Of those trials identified only by handsearching, 252 (68%) were meeting abstracts or published in supplements which MEDLINE had not indexed. 117 (25%) of the 462 RCTs which had a MEDLINE record were missed by the electronic search because they did not have the Publication Type terms RANDOMIZED CONTROLLED TRIAL or CONTROLLED CLINICAL TRIAL.

Conclusions: The overall results indicate that a combination of MEDLINE searching and handsearching is required to identify adequately reports of RCTs, but variation in the relative contributions of the two methods over time will be investigated. This study forms part of a larger research programme into methods of trial identification being undertaken by the UK Cochrane Centre.