Assessment of the evidence for diagnostic tests and strategies: a systematic review of available tools

Mustafa RA1, Wiercioch W2, Falavigna M3, Zhang Y2, Prediger B2, Cheung A4, Ivanova L2, Arevalo-Rodriguez I5, Ventresca M2, Santesso N2, Brozek JL2, Schünemann HJ2
1McMaster University, Canada. University of Missouri-Kansas City, USA, 2McMaster University, Canada, 3McMaster University, Canada. Federal University of Rio Grande do Sul. Porto Alegre, Brazil, 4University of British Columbia, Canada, 5Fundación Universitaria de Ciencias de la Salud, Colombia

Background: The challenges facing systematic reviewers and guideline developers when drawing conclusions and making recommendations about diagnostic tests and strategies (DTS) are considerably different when compared to treatment decisions.

Objectives: To identify, describe and compare all available instruments, checklists, critical appraisal tools, and indices designed for assessing the quality of evidence (QoE) or strength of recommendations (SoR) dealing with diagnostic tests and strategies.

Methods: We conducted a comprehensive systematic search of electronic databases and websites of major international organization to identify state of the art diagnostic guidelines, methods papers, and diagnostic systematic reviews. We also contacted international experts in the field to identify any additional tools.

Results: We identified 45 tools and modifications of existing tools to assess the QoE of DTS. Most tools acknowledge the importance of assessing the QoE and SoR separately. Most tools include individual quality criteria and study design but no tool rates all quality criteria suggested by the GRADE working group. Only two tools explicitly consider factors that increase the confidence in the evidence and only one tool considered publication bias. When moving from evidence to making decisions, patient values and preferences and resources were rarely considered. There is confusion about the terminology that describes the various factors that influence the QoE and SoR.

Conclusions: The GRADE approach is the most complete approach encompassing all domains to evaluate the QoE for diagnostic test accuracy systematic reviews. However, users will benefit from a more detailed guidance about how to judge these domains. Users will also benefit from a better description of GRADE’s framework about using evidence from these systematic reviews to making decisions and developing recommendations about medical tests.