McMaster Premium LiteratUre Service (PLUS) performs well for identifying new studies to be included in updates of Cochrane Collaboration systematic reviews

Article type
Authors
Hemens B1, Brian-Haynes R1
1Health Information Research Unit, Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
Abstract
Background: Nearly 25% of systematic reviews are out of date within two years of publication and ‘‘updating burden’’ is an important problem. The McMaster Premium Literature Service (PLUS) contains a database of pre-appraised, original studies from high impact clinical journals with a small fraction of the entries in major databases. Limiting screening burden by searching PLUS may reduce the time required to update a review. Objectives: We compared the performance of PLUS to that of the Cochrane Trial Registry (CCTR), Medline and EMBASE for locating studies added during an update of a review. We investigated the effect of studies not found in PLUS on review result. Methods: A sample of new studies in updated Cochrane Systematic Reviews was used to establish a reference standard. Searches were performed for each study in PLUS, CCTR, Medline and EMBASE. Where a primary study was not indexed in PLUS, we examined the effect on the review of excluding the study. We compared the result of each review including only new studies in PLUS to the result using only those not found in PLUS (non-PLUS) via ratio of odds ratios (ROR). Results: Ninety-eight updated reviews with meta-analyses were identified and 71 had a meta-analysis that included a study retrieved by PLUS. The relative recall rates for PLUS, CCTR, Medline and EMBASE were 23%, 95%, 90% and 86%, respectively. PLUS contained all new studies for 13 of 71 reviews. No statistically significant difference between PLUS and non-PLUS new studies was found when RORs were pooled across 39 reviews (ROR 0.929; 95% CI, 0.79–1.093). Nineteen updated reviews had no new studies indexed in PLUS. Conclusions: PLUS included less than a quarter of the new studies in Cochrane Review updates but the majority of reviews in our sample were unaffected by the use of PLUS as a sole source of literature. This may be because PLUS captures the most important studies.