Article type
Year
Abstract
Background:
Clinicians often need to evaluate comparative effectiveness of relevant treatment options for their patients. Network meta-analyses (NMAs) provide an important resource for clinicians to guide decision-making; however, these analyses are complex, and interpretation may prove challenging for users.
Objectives:
To evaluate and develop a novel interactive Summary of Findings table (the MATCH-IT tool) for network meta-analysis results, to support clinicians in interpreting and applying comparative effectiveness evidence into decision-making.
Methods:
We performed qualitative user-testing with a convenience sample of practicing physicians working in hospitals and general practice in Norway, Belgium, and Canada. User testing entailed a brief introduction with a clinical scenario, which was a think aloud session with participant-tool-interaction followed by a semi-structured interview. The tests were recorded, transcribed, and analysed using directed content analysis. The results informed the iterative development process of the MATCH-IT tool.
Results:
Five rounds of user-testing, with 26 participants in total, have resulted in four iterations with updates in the tutorial, layout, navigation functionality, and interphase, as well as the practical-issues-section. We have also added an FAQ section. Clinicians who participated in user-testing had been in clinical practice from 0 to 30 years (median 6, IQR 3-11). Most had little or no prior experience with interpretation of NMA results and sparse knowledge about GRADE methodology. Most clinicians perceived MATCH-IT as easy to interpret and navigate and appreciated its ability to provide overview of the evidence, although some of the interactive features proved to be difficult to discover intuitively. They perceived pictograms and inclusion of practical-issues-information as potentially useful features when interacting with patients. Categorization of results with colour coding to indicate both certainty of evidence and relative effectiveness of interventions was appreciated, and filtering functionality allowed participants to efficiently browse and focus in on the interventions believed to be most relevant for a given clinical scenario.
Conclusions:
MATCH-IT shows promise in supporting decision-making for clinicians when presented with results from NMAs. The tool has already been published within Cochrane systematic reviews and clinical practice guidelines.
Patient, public, and/or healthcare consumer involvement: Clinicians were the subject for user testing, ensuring user friendliness for this key consumer group.
Clinicians often need to evaluate comparative effectiveness of relevant treatment options for their patients. Network meta-analyses (NMAs) provide an important resource for clinicians to guide decision-making; however, these analyses are complex, and interpretation may prove challenging for users.
Objectives:
To evaluate and develop a novel interactive Summary of Findings table (the MATCH-IT tool) for network meta-analysis results, to support clinicians in interpreting and applying comparative effectiveness evidence into decision-making.
Methods:
We performed qualitative user-testing with a convenience sample of practicing physicians working in hospitals and general practice in Norway, Belgium, and Canada. User testing entailed a brief introduction with a clinical scenario, which was a think aloud session with participant-tool-interaction followed by a semi-structured interview. The tests were recorded, transcribed, and analysed using directed content analysis. The results informed the iterative development process of the MATCH-IT tool.
Results:
Five rounds of user-testing, with 26 participants in total, have resulted in four iterations with updates in the tutorial, layout, navigation functionality, and interphase, as well as the practical-issues-section. We have also added an FAQ section. Clinicians who participated in user-testing had been in clinical practice from 0 to 30 years (median 6, IQR 3-11). Most had little or no prior experience with interpretation of NMA results and sparse knowledge about GRADE methodology. Most clinicians perceived MATCH-IT as easy to interpret and navigate and appreciated its ability to provide overview of the evidence, although some of the interactive features proved to be difficult to discover intuitively. They perceived pictograms and inclusion of practical-issues-information as potentially useful features when interacting with patients. Categorization of results with colour coding to indicate both certainty of evidence and relative effectiveness of interventions was appreciated, and filtering functionality allowed participants to efficiently browse and focus in on the interventions believed to be most relevant for a given clinical scenario.
Conclusions:
MATCH-IT shows promise in supporting decision-making for clinicians when presented with results from NMAs. The tool has already been published within Cochrane systematic reviews and clinical practice guidelines.
Patient, public, and/or healthcare consumer involvement: Clinicians were the subject for user testing, ensuring user friendliness for this key consumer group.