Objective: The first step in a systematic review is to find all relevant randomised controlled trials (RCTs) in a systematic and reproducible way. Highly sensitive search strategies have been developed to identify RCTs in MEDLINE and EMBASE. The selection of identified trials to include in a review often involves judgement. Our objective was to evaluate if a systematic, reproducible search strategy by two independent reviewers results in the finding of more RCTs than a more simple search strategy by one reviewer.

Methods: We compared the results of the literature search of a previously conducted systematic review on the efficacy of nonsteroidal antiinflammatory drugs (NSAIDs) for low back pain with the results of the systematic literature search for the update of this review for the Cochrane Collaboration.

Results: In the previously conducted systematic review, the literature search was done by one reviewer with a self-made search strategy. 26 RCTs were identified. In the Cochrane review, the literature search and selection were done by two independent reviewers using the highly sensitive search strategies for MEDLINE and EMBASE as recommended by the Cochrane Handbook and the UK Cochrane Centre. The first selection, based on titles and abstracts, could result in inclusion of the study, exclusion of the study, or could be indecisive. Disagreements between the two reviewers were discussed. If the first selection was indecisive or if disagreement persisted, a final decision on inclusion was based on the full article. Disagreements existed in 35 of the 87 MEDLINE studies (40%) and in 25 of the 175 EMBASE studies (14%). Finally, 42 RCTs were identified with this strategy.

Discussion: Our results suggest that selection of studies should be done by (at least) two independent reviewers and that using a sensitive search strategy and inclusion of an EMBASE search might help to identify all relevant studies.