Article type
Year
Abstract
Background: Many clinical and research questions relate to the same patient population (e.g. people receiving dialysis). Developers of guidelines and systematic reviews require efficient and reproducible search strategies for studies of a given population, but even strategies developed by expert searchers have unknown performance.
Objectives: To assess the consistency and performance of populationspecific search strategy segments used in clinical practice guidelines and Cochrane Reviews, with renal populations as proof-of-concept.
Methods: We collected population-specific segments of search strategies published in Cochrane Reviews and clinical guidelines pertaining to patients with glomerular disease, chronic kidney disease, acute kidney injury, kidney transplantation, and dialysis. Our group developed and validated search strategies for studies relevant to these five patient populations. We tested search performance in a database established by manual review of 22 992 full text articles (from 39 journals) tagged by nephrologists as relevant or not relevant to each population.
Results: While aimed at the same population, search strategies varied in content and performance across guidelines and Cochrane Reviews. Our new strategies demonstrated high sensitivities and specificities (90–99%+) in Pubmed, Ovid Medline, and EMBASE. For example, our high sensitivity acute kidney injury strategy for Ovid Medline achieved 97.62% sensitivity, 96.49% specificity, and 38.78% precision. Our high specificity strategy achieved 90.51%, 99.45%, and 80.11%, respectively. The search strategy used in the most authoritative acute kidney injury guideline (KDIGO) achieved 68.71%, 98.79%, and 56.27%. Three randomly selected Cochrane Reviews averaged 58.73%, 99.83%, and 88.29.
Conclusions: We demonstrated that bibliographic databases can be reliably filtered for population of interest using validated search strategies. Our preliminary results show that, across guidelines and Cochrane Reviews, search strategies for retrieving studies of the same renal population use different terms and have inconsistent retrieval performance. Validated strategies improve performance. Full results will be available for the Colloquium.
Objectives: To assess the consistency and performance of populationspecific search strategy segments used in clinical practice guidelines and Cochrane Reviews, with renal populations as proof-of-concept.
Methods: We collected population-specific segments of search strategies published in Cochrane Reviews and clinical guidelines pertaining to patients with glomerular disease, chronic kidney disease, acute kidney injury, kidney transplantation, and dialysis. Our group developed and validated search strategies for studies relevant to these five patient populations. We tested search performance in a database established by manual review of 22 992 full text articles (from 39 journals) tagged by nephrologists as relevant or not relevant to each population.
Results: While aimed at the same population, search strategies varied in content and performance across guidelines and Cochrane Reviews. Our new strategies demonstrated high sensitivities and specificities (90–99%+) in Pubmed, Ovid Medline, and EMBASE. For example, our high sensitivity acute kidney injury strategy for Ovid Medline achieved 97.62% sensitivity, 96.49% specificity, and 38.78% precision. Our high specificity strategy achieved 90.51%, 99.45%, and 80.11%, respectively. The search strategy used in the most authoritative acute kidney injury guideline (KDIGO) achieved 68.71%, 98.79%, and 56.27%. Three randomly selected Cochrane Reviews averaged 58.73%, 99.83%, and 88.29.
Conclusions: We demonstrated that bibliographic databases can be reliably filtered for population of interest using validated search strategies. Our preliminary results show that, across guidelines and Cochrane Reviews, search strategies for retrieving studies of the same renal population use different terms and have inconsistent retrieval performance. Validated strategies improve performance. Full results will be available for the Colloquium.