Background: There has been a steady increase in the number of overviews of systematic reviews published in the last 15 years. The ability to identify published overviews efficiently is of importance to users of evidence syntheses, as well as to researchers interested in investigating their methods. Locating overviews is difficult due to a lack of a validated search strategy and inconsistent terminology used to describe overviews.
Objectives: To develop a validated search strategy to retrieve overviews of systematic reviews in MEDLINE.
Methods: We derived a 'gold standard' test set of overviews published between 1998 and 2011 from the references of two methods papers. Two population sets were used to identify discriminating terms; that is, terms that appear frequently in the test set, but infrequently in two population sets of references found in MEDLINE. We used a text mining package in the statistical program R to conduct a frequency analysis of terms appearing in the titles and abstracts. Terms that had a sensitivity of 10% or more in the test set and a corresponding low sensitivity (less than 2%) in the two population sets were selected for further testing. Candidate terms were combined in search strategies and tested in MEDLINE. Sensitivity and precision were used to evaluate filter performance. We evaluated the performance of the strategy against an externally derived set of overviews published between 2012 and 2014.
Results: Two search strategies were developed. The sensitivity-maximising strategy offers 93% sensitivity in retrieving overviews of reviews, with a low level of precision (4%). The sensitivity-and-precision-maximising strategy offers a less sensitive strategy (66%), but a higher precision (21%).
Conclusions: We have developed two validated search strategies for locating overviews of systematic reviews using a text mining approach. Consistent language used to describe overviews of reviews would aid in their identification, as would a specific MEDLINE publication type.