Assessing how much certainty to place in findings from qualitative evidence syntheses: the CerQual approach

Tags: Oral
Lewin S1, Glenton C2, Munthe-Kaas H3, Carlsen B4, Colvin C5, Noyes J6, Rashidian A7
1Norwegian Knowledge Centre for the Health Services and MRC South Africa, Norway, 2Norwegian Branch of the Nordic Cochrane Centre, Norwegian Knowledge Centre for the Health Services, Norway, 3Norwegian Knowledge Centre for the Health Services, Norway, 4Uni Rokkan Centre, Norway, 5Centre for Infectious Disease Epidemiology and Research (CIDER), University of Cape Town, South Africa, 6Centre for Health-Related Research, Bangor University, UK, 7National Institute of Health Research and School of Public Health, Tehran University of Medical Sciences, Iran

Background: Qualitative evidence syntheses are increasingly used to bring together findings from qualitative studies. However, it is difficult to use these findings alongside Cochrane effects reviews, or to inform policy development, because methods to assess how much certainty to place in these synthesis findings are poorly developed.

Objectives: To describe a novel approach for assessing how much certainty (or confidence) to place in the findings of qualitative evidence syntheses.

Methods: The certainty of the qualitative evidence (CerQual) approach was developed through review of existing tools in this area; discussionswithin aworking group; and piloting of the tool on three qualitative evidence syntheses.

Results: The CerQual approach bases assessments of certainty on two factors: the methodological limitations of the individual studies contributing to a review finding and the coherence of each review finding. Methodological limitations are assessed using a quality-assessment tool for qualitative studies. Coherence is assessed by looking at the extent to which it is possible to identify a clear pattern across the individual study data. Coherence may be further strengthened if the individual studies contributing to the finding are drawn from a wide range of settings. We propose three levels of certainty: high, moderate and low. Findings drawn from generally well-conducted studies with few methodological limitations—and showing high levels of coherence—are rated as ‘high’ certainty. Findings are assessed as ‘moderate’ certainty where there are concerns regarding either the methodological limitations of the studies or the coherence of the review finding. Where the studies have important methodological limitations and there are concerns regarding the coherence of the review finding, the certainty is assessed as ‘low’.

Conclusions: The CerQual approach provides a transparent method for assessing the certainty of evidence from qualitative syntheses and may facilitate the use of these findings alongside Cochrane Reviews of effects.