Peer review of literature search strategies: does it make a difference?

Spry C1, Mierzwinski-Urban M1, Rabb D1
1Canadian Agency for Drugs and Technologies in Health Canada (CADTH), Canada

Background: Peer review is an integral part of scientific research. For Information Specialists, peer review feedback is used to validate the quality of search strategies.

Objectives: To determine whether the peer review of literature search strategies has an effect on the number and quality of articles included in Canadian Agency for Drugs and Technologies in Health (CADTH) rapid review reports.

Methods: A total of 150 pairs of pre-peer-reviewed and peer-reviewed search strategies for CADTH rapid review reports related to health devices medical procedures, and pharmaceuticals were randomly selected and screened. For search strategies meeting specified selection criteria, pre-peer-reviewed and their corresponding peer-reviewed searches were run and the search results were compared. Unique articles retrieved solely by peer-reviewed searches and included in the final reports were identified and categorized according to publication type.

Results: Of the 150 pairs of pre-peer-reviewed and peer-reviewed searches screened, 47 met the selection criteria. Of these 47 pairs of searches, 43% (20/47) of the peer-reviewed searches retrieved a total of 81 unique articles that were included in rapid review reports. The 81 articles consisted of 5 systematic reviews, 3 randomized controlled trials, 52 non-randomized studies, 1 guideline, 18 review articles and 2 other.

Conclusions: The results of this study suggest that the peer review of literature search strategies can improve both the number and quality of relevant articles retrieved.