Article type
Year
Abstract
Background:
The Cochrane Handbook for Systematic Reviews of Interventions contains two search filters to find randomised controlled trials (RCT) in Ovid MEDLINE: a sensitivity maximising RCT filter and a sensitivity and precision maximising RCT filter. The RCT search strategies were originally published in 1994 and have been adapted and updated, most recently in 2008.
Objectives:
To determine whether the Cochrane filters are still performing adequately to inform Cochrane reviews, we tested the performance of the two Cochrane RCT filters and 36 other MEDLINE filters in a very large gold standard set of RCT records.
Methods:
We identified a gold standard set of 27,617 RCT reports published in 2016 from the Cochrane Central Register of Controlled Trials (CENTRAL) database. We then retrieved these RCT records in Ovid MEDLINE using PubMed identifiers. Each RCT filter was run in MEDLINE and combined with gold standard set of records, to determine its sensitivity, precision and f-score.
Results:
We tested each filter against the gold standard on 16 July 2019. The most sensitive RCT filter was Duggan (sensitivity 0.99). The Cochrane sensitivity maximising RCT filter had a sensitivity of 0.96, but was more precise than Duggan (0.14 compared to 0.04 for Duggan). The most precise RCT filters were Chow, Glanville/Lefebvre, Royle/Waugh, Dumbrique (precision 0.97, sensitivity 0.83). The best precision Cochrane filter was the sensitivity and precision maximising RCT filter (precision 0.46).
Conclusions:
This study used a very large gold standard to compare the performance of all known RCT filters. We concluded that the Cochrane MEDLINE sensitivity maximising RCT filter can continue to be used with confidence by Cochrane reviewers and CENTRAL compilers. Slightly more sensitive filters are available, but with lower precision. Using the information from this research, searchers can make better informed decisions about which filters to use for their own searches.
Patient or healthcare consumer involvement: This was a searching methods investigation and no patients or healthcare consumers were involved.
The Cochrane Handbook for Systematic Reviews of Interventions contains two search filters to find randomised controlled trials (RCT) in Ovid MEDLINE: a sensitivity maximising RCT filter and a sensitivity and precision maximising RCT filter. The RCT search strategies were originally published in 1994 and have been adapted and updated, most recently in 2008.
Objectives:
To determine whether the Cochrane filters are still performing adequately to inform Cochrane reviews, we tested the performance of the two Cochrane RCT filters and 36 other MEDLINE filters in a very large gold standard set of RCT records.
Methods:
We identified a gold standard set of 27,617 RCT reports published in 2016 from the Cochrane Central Register of Controlled Trials (CENTRAL) database. We then retrieved these RCT records in Ovid MEDLINE using PubMed identifiers. Each RCT filter was run in MEDLINE and combined with gold standard set of records, to determine its sensitivity, precision and f-score.
Results:
We tested each filter against the gold standard on 16 July 2019. The most sensitive RCT filter was Duggan (sensitivity 0.99). The Cochrane sensitivity maximising RCT filter had a sensitivity of 0.96, but was more precise than Duggan (0.14 compared to 0.04 for Duggan). The most precise RCT filters were Chow, Glanville/Lefebvre, Royle/Waugh, Dumbrique (precision 0.97, sensitivity 0.83). The best precision Cochrane filter was the sensitivity and precision maximising RCT filter (precision 0.46).
Conclusions:
This study used a very large gold standard to compare the performance of all known RCT filters. We concluded that the Cochrane MEDLINE sensitivity maximising RCT filter can continue to be used with confidence by Cochrane reviewers and CENTRAL compilers. Slightly more sensitive filters are available, but with lower precision. Using the information from this research, searchers can make better informed decisions about which filters to use for their own searches.
Patient or healthcare consumer involvement: This was a searching methods investigation and no patients or healthcare consumers were involved.