Systematic reviews of complex interventions: From bare bones to greater policy relevance

Article type
Authors
Snilstveit B1, Waddington H1
1International Initiative for Impact Evaluation (3ie), UK
Abstract
Background: Systematic reviews have traditionally focused on synthesising quantitative evidence, typically from randomised controlled trials. For questions regarding the effectiveness of simple interventions, such as effectiveness of a drug, 'bare bones reviews' are invaluable. However, when applied to systematic reviews of complex interventions the bare bones approach to systematic reviewing has a number of limitations, including its usefulness for informing policy and practice. This has been particularly highlighted by the recent rise in demand for systematic reviews in the field of international development.

Objectives: Outline an approach to systematic reviewing that is appropriate for reviews of complex interventions, with the aim of enhancing the usefulness of such reviews for policy and practice.

Methods: Based on a review of the methodological literature in this field and the experience of 3ie’s systematic reviews program, this paper provides an outline 3ie’s methodological approach to systematic reviews.

Results: Theory-based systematic reviews, which summarise evidence on what works, when and why, can improve the policy relevance of systematic reviews of complex interventions. This approach situates reviews in the broader context of the underlying programme theory, reporting and synthesising evidence on all assumptions and links in the causal chain. Reviews that answer these questions draw on a range of study types. Answering the what works and what doesn’t questions means drawing on effectiveness studies, conducted to standards of high quality impact evaluation. But formulating answers to the when and why questions, may require a broader range of evidence from both quantitative and qualitative research.

Conclusions: Systematic reviews of complex interventions should draw on a program theory, and when appropriate and feasible, use a mixed methods approach to collect and synthesise evidence along the causal chain.