Article type
Year
Abstract
Background: There is a growing recognition amongst qualitative evidence synthesis (QES) experts that meta-ethnography requires conceptually rich studies, as more descriptive studies usually have insufficient depth for an interpretive synthesis. However, understandings of ‘conceptual richness’ are diverse and often vague, and there is currently no established method for its assessment. We therefore developed and applied an approach for assessing conceptual richness for a Cochrane QES on acceptance of childhood vaccination.
Method: Our approach drew on Sandelowski and Barroso’s (2007) typology of the type and nature of qualitative findings. This typology conceives qualitative findings as being located along a spectrum based on the degree of data abstraction or ‘transformation’. At one end of the spectrum are more descriptive findings that describe patterns in the data. At the more transformative end of the spectrum are interpretive or explanatory findings. These findings have a high-level of abstraction and provide theoretical interpretations or explanations, often across multiple patterns within the data. We created a 5-point scale to categorise studies on this spectrum and developed clear definitions for each score. We agreed that studies with a score of ≥3 would be included in our QES.
Results: A total of 136 studies (165 articles) met the inclusion criteria for our QES and were assessed for conceptual richness using our 5-point scale. Initially, 25 random studies were scored independently by two investigators who were both experienced qualitative researchers. Thereafter, one investigator performed the assessment on the remaining eligible studies, a sample of which were checked by a second investigator. Both investigators experienced challenges with the process, and comparison of their assessments revealed some variation. On reflection, several strategies may have helped reduce these challenges and variations. These include identifying a ‘prototype’ study for each score before beginning the assessment process to serve as a reference point; reducing the scale to 3-points; refining our definition of conceptual richness to incorporate relevance to the synthesis objectives and to more clearly distinguish it from methodological quality; and developing a more systematic approach for gauging overall ranking for studies with multiple articles of varying richness. The value of having two (or more) investigators may be less about achieving inter-rater reliability and more about facilitating the making, challenging and articulating of what are often far from straightforward judgements.
Conclusion: While assessments of conceptual richness are partly intuitive and subjective, there is a need for greater debate and transparency regarding how we define and judge richness. Our method provides one potentially fruitful approach which could serve as a foundation for further work and practical application.
Patient or healthcare consumer involvement: None
Method: Our approach drew on Sandelowski and Barroso’s (2007) typology of the type and nature of qualitative findings. This typology conceives qualitative findings as being located along a spectrum based on the degree of data abstraction or ‘transformation’. At one end of the spectrum are more descriptive findings that describe patterns in the data. At the more transformative end of the spectrum are interpretive or explanatory findings. These findings have a high-level of abstraction and provide theoretical interpretations or explanations, often across multiple patterns within the data. We created a 5-point scale to categorise studies on this spectrum and developed clear definitions for each score. We agreed that studies with a score of ≥3 would be included in our QES.
Results: A total of 136 studies (165 articles) met the inclusion criteria for our QES and were assessed for conceptual richness using our 5-point scale. Initially, 25 random studies were scored independently by two investigators who were both experienced qualitative researchers. Thereafter, one investigator performed the assessment on the remaining eligible studies, a sample of which were checked by a second investigator. Both investigators experienced challenges with the process, and comparison of their assessments revealed some variation. On reflection, several strategies may have helped reduce these challenges and variations. These include identifying a ‘prototype’ study for each score before beginning the assessment process to serve as a reference point; reducing the scale to 3-points; refining our definition of conceptual richness to incorporate relevance to the synthesis objectives and to more clearly distinguish it from methodological quality; and developing a more systematic approach for gauging overall ranking for studies with multiple articles of varying richness. The value of having two (or more) investigators may be less about achieving inter-rater reliability and more about facilitating the making, challenging and articulating of what are often far from straightforward judgements.
Conclusion: While assessments of conceptual richness are partly intuitive and subjective, there is a need for greater debate and transparency regarding how we define and judge richness. Our method provides one potentially fruitful approach which could serve as a foundation for further work and practical application.
Patient or healthcare consumer involvement: None