Article type
Year
Abstract
Background: The cornerstone of a valid systematic review is a comprehensive literature search. Several search filters have been developed and are advocated to identify primary studies for diagnostic accuracy reviews. Published values for sensitivity and specificity of these filters range from 53-100% and 74-98% respectively, however their validity for Cochrane systematic reviews is uncertain because of limited inclusion criteria (generally only the major biomedical journals), or because the reference standard to validate these filters, handsearching, was not used.
Objectives: To evaluate the performance of published search strategies for finding SDTAs in Medline and Embase to inform Collaboration policy decisions regarding appropriate methods to conduct diagnostic test reviews and develop a diagnostic test register.
Methods: All issues from the three top ranking kidney journals for 1990-1991 and 2002-2003 were handsearched by two authors to identify all SDTA using STARD criteria, irrespective of quality, with disagreements resolved by a third person. The resulting set of articles was entered into the Ovid Medline and Embase databases forming the reference standard. Nine filters, the most sensitive, specific and/or precise from all relevant published papers, were entered into Medline and each run against this reference standard, and values for sensitivity, specificity and precision were calculated. Four filters were adapted for Embase, following the same process.
Results: From 4,409 articles, 100 were identified as SDTAs. In Medline, sensitivity ranged from 37-83%, specificity from 67-97% and precision from 5-23%. In Embase the corresponding values were sensitivity 43-84%, specificity 80-97% and precision 8.5-24.5%. All values were lower than those reported in the published papers. The primary reason for missed studies was inadequate indexing of methodological terms.
Conclusions: None of the filters tested in Medline or Embase performed satisfactorily as a means of developing a comprehensive register of kidney-related SDTAs. A larger study across more health areas needs to be done to confirm these findings. Based upon these data, only handsearching, and retagging in a specific field for subsequent electronic searching, is sufficiently comprehensive for the purposes of systematic reviews.
Objectives: To evaluate the performance of published search strategies for finding SDTAs in Medline and Embase to inform Collaboration policy decisions regarding appropriate methods to conduct diagnostic test reviews and develop a diagnostic test register.
Methods: All issues from the three top ranking kidney journals for 1990-1991 and 2002-2003 were handsearched by two authors to identify all SDTA using STARD criteria, irrespective of quality, with disagreements resolved by a third person. The resulting set of articles was entered into the Ovid Medline and Embase databases forming the reference standard. Nine filters, the most sensitive, specific and/or precise from all relevant published papers, were entered into Medline and each run against this reference standard, and values for sensitivity, specificity and precision were calculated. Four filters were adapted for Embase, following the same process.
Results: From 4,409 articles, 100 were identified as SDTAs. In Medline, sensitivity ranged from 37-83%, specificity from 67-97% and precision from 5-23%. In Embase the corresponding values were sensitivity 43-84%, specificity 80-97% and precision 8.5-24.5%. All values were lower than those reported in the published papers. The primary reason for missed studies was inadequate indexing of methodological terms.
Conclusions: None of the filters tested in Medline or Embase performed satisfactorily as a means of developing a comprehensive register of kidney-related SDTAs. A larger study across more health areas needs to be done to confirm these findings. Based upon these data, only handsearching, and retagging in a specific field for subsequent electronic searching, is sufficiently comprehensive for the purposes of systematic reviews.