The challenge of assessing quality of evidence generated from mixed treatment comparisons and a potential solution

Article type
Authors
Soares M1, O’Meara S2, Cullum N2
1Centre for Health Economics, University of York, UK
2Department of Health Sciences, University of York, UK
Abstract
Background: Mixed treatment comparisons (MTCs) allow the synthesis of direct and indirect effectiveness evidence and can be used, even when available data are limited, to help healthcare decision-makers interpret all available evidence. However, ensuring that estimates from these ’sparse’ MTCs are interpreted correctly requires the development of appropriate quality of evidence assessments. We applied a MTC approach to the evidence for wound dressings for diabetic foot ulcers. In order to produce a 'Summary of Findings' table we attempted to adapted GRADE to allow for the MTC.

Objectives: i) to determine the relative effects of alternative wound dressings on diabetic foot ulcer healing; ii) to assess how to reflect the volume and quality of research evidence in a MTC systematic review.

Methods: Systematic literature searches and Bayesian mixed treatment comparison with sensitivity analyses and assessment of inconsistencies. The GRADE evidence rating approach was applied to direct network links and a modified GRADE approach applied to MTC evidence estimates based on: the limitations of the informing studies; impact of sensitivity analyses; the volume of direct and indirect data and data consistency; uncertainty in the resulting estimates and publication bias related to informing studies.

Results: Fifteen RCTs (nine dressings) were included (Figure 1). Three dressings estimated as having the highest probabilities of being the 'best’ for ulcer healing had the lowest quality direct evidence. Four dressings informed by higher quality evidence were estimated to have no probability of being the best dressing. A modified GRADE approach for MTCs helped systematically assess and communicate the quality of this MTC-derived evidence to aid interpretation.

Conclusions: MTCs are an important evidence synthesis tool likely to be increasingly incorporated into systematic reviews and overviews of reviews. Quality assessment methods for such outputs are required. We suggest that adpated GRADE evidence quality rating methods should be considered for this purpose.