Article type
Abstract
Background
Clinical Decision Support (CDS) systems support healthcare providers and their patients in the process of clinical decision-making. Clinical practice guidelines and recommendations are combined with patient-specific data to propose one or several patient tailored treatment options.
Cebam, the Belgian Center of Evidence Based Medicine, develops evaluation criteria and assesses the methodological trustworthiness of guidelines and non-guideline evidence-based practice (EBP) sources.
Objectives
Based on our experience with the assessment of guidelines and EBP-sources, we aimed to develop a tool for the assessment of the trustworthiness of CDS-systems.
Methods
In a CDS-system, recommendations are converted into so-called clinical decision rules. These are decision models or processes such as tree-like models of decisions consisting of multiple steps of ‘if then else’ logic. The evaluation of the trustworthiness of a CDS-system can be divided in three main steps: (1) Evaluation of the guidelines or evidence-based recommendations used as source information to develop decision rules; (2) Evaluation of the methods used to translate recommendations into decision rules; (3) Evaluation of the decision rules themselves. A systematic search was executed in Medline to identify existing tools for evaluation of trustworthiness of decision rules. After critical appraisal, one tool was selected for further use. A team of methodologists experienced in guideline and EBP-source assessment defined in consensus the criteria for step 1, 2 and 3, based on the retrieved tool and their own expertise.
Results
For the evaluation of step 1 and similar to the assessment criteria used by Cebam, the validated CAPOCI-tool was selected for evaluation of non-guideline EBP-sources and the AGREE II instrument was selected for the evaluation of guidelines. For the evaluation of step 2 and 3, additional criteria were defined for evaluating the process used to convert recommendations into decision rules. For the evaluation of step 3, 3 criteria were defined based on the GUIDES-checklist for evaluating the content of a CDS-system.
Conclusion
The developed criteria and procedure may support validation teams in the assessment of the trustworthiness of CDS-systems. It may also provide guidance to developers of CDS-systems and present an added value to an evidence based clinical decision-making process.
Clinical Decision Support (CDS) systems support healthcare providers and their patients in the process of clinical decision-making. Clinical practice guidelines and recommendations are combined with patient-specific data to propose one or several patient tailored treatment options.
Cebam, the Belgian Center of Evidence Based Medicine, develops evaluation criteria and assesses the methodological trustworthiness of guidelines and non-guideline evidence-based practice (EBP) sources.
Objectives
Based on our experience with the assessment of guidelines and EBP-sources, we aimed to develop a tool for the assessment of the trustworthiness of CDS-systems.
Methods
In a CDS-system, recommendations are converted into so-called clinical decision rules. These are decision models or processes such as tree-like models of decisions consisting of multiple steps of ‘if then else’ logic. The evaluation of the trustworthiness of a CDS-system can be divided in three main steps: (1) Evaluation of the guidelines or evidence-based recommendations used as source information to develop decision rules; (2) Evaluation of the methods used to translate recommendations into decision rules; (3) Evaluation of the decision rules themselves. A systematic search was executed in Medline to identify existing tools for evaluation of trustworthiness of decision rules. After critical appraisal, one tool was selected for further use. A team of methodologists experienced in guideline and EBP-source assessment defined in consensus the criteria for step 1, 2 and 3, based on the retrieved tool and their own expertise.
Results
For the evaluation of step 1 and similar to the assessment criteria used by Cebam, the validated CAPOCI-tool was selected for evaluation of non-guideline EBP-sources and the AGREE II instrument was selected for the evaluation of guidelines. For the evaluation of step 2 and 3, additional criteria were defined for evaluating the process used to convert recommendations into decision rules. For the evaluation of step 3, 3 criteria were defined based on the GUIDES-checklist for evaluating the content of a CDS-system.
Conclusion
The developed criteria and procedure may support validation teams in the assessment of the trustworthiness of CDS-systems. It may also provide guidance to developers of CDS-systems and present an added value to an evidence based clinical decision-making process.