Assessing how much confidence to place in findings from qualitative evidence syntheses: a new version of the GRADE-CERQual tool

Tags: Oral
Glenton C1, Lewin S2, Booth A3, Noyes J4, Garside R5, Tuncalp O6, Carlsen B7, Bohren M6, Wainwright M8, Rashidian A9, Colvin C8, Munthe-Kaas H1
1Norwegian Knowledge Centre for the Health Services, Norway, 2Norwegian Knowledge Centre for the Health Services; Medical Research Council of South Africa, Norway, 3University of Sheffield, United Kingdom, 4Bangor University, United Kingdom, 5University of Essex Medical School, United Kingdom, 6World Health Organization, Switzerland, 7Uni Research Rokkan Centre, Norway, 8University of Cape Town, South Africa, 9Tehran University of Medical Sciences, Iran

Background: Systematic reviews of qualitative studies (qualitative evidence syntheses) are increasingly used to bring together findings from qualitative studies. In order to use the synthesised findings to inform decisions we need methods to assess how much confidence to place in these findings.

Objectives: To describe the latest version of a Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool for assessing how much confidence to place in findings from qualitative evidence syntheses.

Methods: GRADE’s Confidence of the Evidence from Reviews of Qualitative research (CERQual) tool was developed through review of existing tools; working group discussions; and piloting of the tool on qualitative evidence syntheses.

Results: CERQual bases assessments of confidence on four components:

1. the methodological limitations of the individual studies contributing to a review finding, assessed using a quality-assessment tool for qualitative studies;

2. the coherence of each review finding, assessed by looking at the extent to which a review finding is based on data that are similar within and across multiple individual studies and/or incorporates convincing explanations for any variations across individual studies;

3. the relevance of a review finding, determined by the extent to which the primary studies supporting a review finding are applicable to the context specified in the review question; and

4. the adequacy of data supporting a review finding, assessed by an overall determination of the degree of richness or scope of the evidence and quantity of data supporting a review finding.

After assessing each component, an overall judgement of the confidence in each review finding is made. Confidence is assessed as high, moderate, low, or very low.

Conclusions: GRADE-CERQual provides a transparent method for assessing the confidence of evidence from reviews of qualitative research. Like other GRADE tools, it may facilitate the use of these findings alongside reviews of effects, in Health Technology Assessments and in guideline development processes.