Article type
Abstract
Background: There is little consensus on how best to assess methodological limitations in the context of qualitative evidence syntheses (QES). Current critical appraisal tools are neither specifically developed for use in Cochrane reviews or GRADE-CERQual, nor are they evidence-based. The aim of CochrAne qualitative MEthodological LimitatiOns Tool (CAMELOT) was to address this gap by using consensus methods to develop an evidence-based tool that uses qualitative principles to assess methodological limitations of primary qualitative research in the context of QES and GRADE-CERQual.
Objectives: To develop a novel, evidence- and consensus-based approach for assessing methodological strengths and limitations of primary qualitative research that is specifically intended for use with Cochrane QES and GRADE-CERQual.
Methods: We developed CAMELOT by (1) identifying existing critical appraisal tools and synthesizing included criteria (published systematic review; Munthe-Kaas et al 2020), (2) identifying evidence to support inclusion of criteria in a critical appraisal tool, (3) consensus survey to agree on CAMELOT domains and definitions and (4) interviews to explore user experience with CAMELOT.
Results: CAMELOT is comprised of twelve domains: four Meta domains (Research aim & question(s), Researchers, Stakeholders, Context) and eight Method domains (Research approach, Ethical considerations, Equity, diversity / inclusion considerations, Theory, Participant recruitment & selection, Data collection, Analysis and interpretation, Presentation of findings). Meta domains encourage review authors to consider characteristics of the primary study that go beyond how the study was carried out but that inform the conduct and design of the study. Method domains encourage review authors to consider how the study was designed, planned and/or conducted, and how study conduct and design fits with the information provided in the four Meta domains. In the context of CAMELOT, appropriateness of fit refers to the degree to which the design and conduct of the study is suitable for the research aim and/or question(s), aligns with the researchers and/or stakeholders involved in the study, and/or is suitable given the study context.
Conclusions: CAMELOT is a novel approach to support the assessment of methodological strengths and limitations of qualitative research in QES.
Objectives: To develop a novel, evidence- and consensus-based approach for assessing methodological strengths and limitations of primary qualitative research that is specifically intended for use with Cochrane QES and GRADE-CERQual.
Methods: We developed CAMELOT by (1) identifying existing critical appraisal tools and synthesizing included criteria (published systematic review; Munthe-Kaas et al 2020), (2) identifying evidence to support inclusion of criteria in a critical appraisal tool, (3) consensus survey to agree on CAMELOT domains and definitions and (4) interviews to explore user experience with CAMELOT.
Results: CAMELOT is comprised of twelve domains: four Meta domains (Research aim & question(s), Researchers, Stakeholders, Context) and eight Method domains (Research approach, Ethical considerations, Equity, diversity / inclusion considerations, Theory, Participant recruitment & selection, Data collection, Analysis and interpretation, Presentation of findings). Meta domains encourage review authors to consider characteristics of the primary study that go beyond how the study was carried out but that inform the conduct and design of the study. Method domains encourage review authors to consider how the study was designed, planned and/or conducted, and how study conduct and design fits with the information provided in the four Meta domains. In the context of CAMELOT, appropriateness of fit refers to the degree to which the design and conduct of the study is suitable for the research aim and/or question(s), aligns with the researchers and/or stakeholders involved in the study, and/or is suitable given the study context.
Conclusions: CAMELOT is a novel approach to support the assessment of methodological strengths and limitations of qualitative research in QES.