Background: It is difficult to keep up with new information for both performing new reviews and updating existing reviews. Indeed, it is estimated that more than half of the systematic reviews in the Cochrane Library have not been updated for at least 2 years. One reason for this is the exponential growth of the biomedical literature; there is simply too much literature to keep up.

Objectives: To demonstrate that modern data mining tools can be one step in reducing the labor necessary to produce and maintain systematic reviews.

Methods: We used four continuously updated, manually-curated resources that summarize MEDLINE-indexed articles in entire fields using systematic review methods (PDGene, AlzGene, SzGene for genetic determinants of Parkinson’s, Alzheimer’s and schizophrenia, respectively; and the Tufts CEARegistryfor cost-effectiveness analyses). In each dataset, we trained a classification model on citations screened up to 2009. We then evaluated its ability to classify citations published in 2010 as ‘relevant’ or ‘irrelevant’, using human screening as the gold standard.

Results: Classification models did not miss any of the 104, 65 and 179 eligible citations in PDGene, AlzGene and SzGene, respectively, and missed only 1 out of 79 in CEA Registry (100% sensitivity for the three first, 99% for the fourth). The respective specificities were 90%, 93%, 90% and 73%. Had the semi-automated system been used in 2010, a human would have needed to read only 605/5616 citations to update the PDGene registry (11%) and 555/7298 = 8%, 717/5381=13% and 334/1015=33% for the other three databases.

Conclusions: Data mining methodologies can reduce the burden of updating systematic reviews, without missing more papers than humans.