A search strategy for prognostic reviews and for reviews on diagnostic and prognostic prediction models

Article type
Authors
Moons K1, Moons C1, Zuithoff P1, Vergouwe Y1
1Julius Center, UMC Utrecht, Utrecht, Utrecht, Netherlands
Abstract
Background: Prognostic reviews are gradually increasing, including reviews on prognostic and diagnostic risk or prediction models. To properly review existing evidence, an accurate search strategy is needed. Objective: We validated and updated a previously introduced search strategy for multivariable prediction models, for its ability to indentify studies on the development, validation, impact or any other evaluation of a diagnostic or prognostic prediction model, regardless the context. We also studied how to update the search strategy to include non-multivariable prognostic factor studies. Methods: We used PUBMED to indentify all prediction studies from six general journals in 2008 (Annals of Internal Medicine, BMJ, Lancet, NEJM, PLos Medicine, JAMA), by applying the search strategy. The so-identified studies were compared by a complete hand-search of these journals by two reviewers, which was considered as the reference standard. We included all studies which developed, validated or otherwise evaluated a prognostic or diagnostic risk score/model with at least 2 predictors. We calculated the accuracy (sensitivity and specificity) of the search strategy. When writing this abstract, three journals were yet fully hand searched. Results: The hand search revealed 1542 hits (i.e. all publications in the three journals, excluding publication types like comments or editorials). The search strategy revealed 172 hits. After abstract and full text screening, 29 were identified as prediction studies. The hand search revealed 31, i.e. 2 false negatives by the search strategy (both impact studies); sensitivity = 94%. 1368 hits were true negative by the search-strategy and 143 false positive; specificity = 91% (1368/1511). The search strategy missed 8 studies searching for new (independently associated) predictors; sensitivity = 74%, which we considered too low. The strategy will be updated for use for prognostic study reviews in general. Conclusion: Application of the search strategy results in a very low number missed diagnostic and prognostic prediction model studies, with a relatively low number of false positives.