Systematic reviews of the effectiveness of complex interventions: some methodological issues

Pantoja T1, Grimshaw J2
1Department of Family Medicine & Health Policy and Systems Research Unit, School of Medicine, Pontificia Universidad Catolica de Chile, Chile, 2Co-ordinating Editor Cochrane Effective Practice and Organisation of Care Group, Canada

Background: Complex interventions—usually describe as interventions that contains several interacting components—are widely used in health services and in other areas of social policy that have important health consequences such as education, transport and housing. Well conducted systematic reviews are increasingly seen as providing the best evidence to guide choices about this kind of interventions. But in contrast with reviews of pharmacological interventions, a number of methodological challenges should be addressed by reviewers undertaking reviews of complex interventions.

Objective: To discuss methodological issues relevant to the conduct of reviews of effectiveness of complex interventions.

Methods: We build on our experiences within the Cochrane Effective Practice and Organisation of Care (EPOC) group, discussions at a recently held workshop on the issue, and a focused review of the methodological literature in the area.

Results: We have selected the following issues for a further analysis: (i) Framing the review: specification of the intervention, the use of logic models/theoretical frameworks; and the broadness/narrowness of the review question (the ‘lumping versus splitting’ debate). (ii) Dealing with heterogeneity: what is too much variability to make meta-analysis invalid?; when additional methods are needed?; what are the potential analytical strategies?. (iii) Methods of synthesis/analysis: appropriateness of meta-analysis; other synthesis approaches (range of effects size); incorporating different sources of evidence; identifying effect modifiers. (iv) Applicability: what are the ‘active’ intervention components?; what intervention was implemented in primary studies (‘implementation fidelity’)?.

Conclusions: We have discussed issues that reviewers will face when conducting reviews of the effectiveness of complex interventions. Additionally we have presented some of the current methodological developments in order to contribute to more sound and informative reviews in this field.