Article type
Year
Abstract
Background:
Systematic reviews addressing policy and practice questions concerning complex interventions frequently need to both assess the efficacy of a given intervention and to identify which intervention and its components might be relevant in particular circumstances. Here, intervention replication is rare, and commonly used synthesis methods are less useful when analyses focuses on the identification of those components of an intervention which are critical to its success.
Methods:
Through a worked example, we examine the potential of Qualitative Comparative Analysis (QCA) to assist with complex syntheses. Originating from political science and historical sociology, QCA can identify a number of configurations of various participant, intervention, and contextual characteristics which are (or are not) present when the intervention has been successful (or not) in obtaining the outcome. Analysing studies in these terms facilitates the identification of necessary and sufficient conditions to obtain the outcome. Since QCA is predicated on the assumption that multiple pathways might lead to the same outcome, and does not assume a linear additive model in changes to a particular condition, it appears less sensitive to some limitations of statistical meta-analytic methods.
Results:
The worked example shows how the QCA reveals our initial theories of change to be unable to distinguish between ‘effective’ and ‘highly effective’ interventions. Through the iterative QCA process, other intervention characteristics are identified which better explain the observed results.
Conclusions:
QCA is a promising alternative (or adjunct) synthesis method, particularly in comparison to the standard fall-back of a ‘narrative synthesis’ when quantitative synthesis is impossible, and might be considered when reviews are broad and heterogeneity is significant. There are currently very few examples of its use with systematic review data, and further methodological work is needed to establish optimal conditions for its use and to document process, practice, and reporting standards.
Systematic reviews addressing policy and practice questions concerning complex interventions frequently need to both assess the efficacy of a given intervention and to identify which intervention and its components might be relevant in particular circumstances. Here, intervention replication is rare, and commonly used synthesis methods are less useful when analyses focuses on the identification of those components of an intervention which are critical to its success.
Methods:
Through a worked example, we examine the potential of Qualitative Comparative Analysis (QCA) to assist with complex syntheses. Originating from political science and historical sociology, QCA can identify a number of configurations of various participant, intervention, and contextual characteristics which are (or are not) present when the intervention has been successful (or not) in obtaining the outcome. Analysing studies in these terms facilitates the identification of necessary and sufficient conditions to obtain the outcome. Since QCA is predicated on the assumption that multiple pathways might lead to the same outcome, and does not assume a linear additive model in changes to a particular condition, it appears less sensitive to some limitations of statistical meta-analytic methods.
Results:
The worked example shows how the QCA reveals our initial theories of change to be unable to distinguish between ‘effective’ and ‘highly effective’ interventions. Through the iterative QCA process, other intervention characteristics are identified which better explain the observed results.
Conclusions:
QCA is a promising alternative (or adjunct) synthesis method, particularly in comparison to the standard fall-back of a ‘narrative synthesis’ when quantitative synthesis is impossible, and might be considered when reviews are broad and heterogeneity is significant. There are currently very few examples of its use with systematic review data, and further methodological work is needed to establish optimal conditions for its use and to document process, practice, and reporting standards.