Heterogeneity and complexity within reviews of social research: the perspective of ‘divergent’ reviews

Article type
Authors
Thomas J1
1Institute of Education, EPPI-Centre, London, UK
Abstract
Background: Policymakers often begin with a particular problem (e.g. an imperative to reduce obesity) and look to research to tell them which interventions are effective in impacting on that problem. Systematic reviews conducted to inform this process thus often start from a given, ‘known’, outcome but do not pre-specify their populations or type of intervention. Such reviews are further complicated in social research because interventions are often multi-faceted, and it is difficult to identify the ‘active ingredients’ that make them effective in any specific case. Reviewers must therefore mediate between the diverse and uneven nature of the research evidence they identify and the need to produce useful and useable findings for their users. Methods: A long-standing programme of systematic reviews for a Government department will be used as a case study. Results: The key challenge has been to identify ways of explaining differences between the results of primary studies. To this end, ‘qualitative’ evidence from those closest to the issues being explored and data from process evaluations has been synthesised alongside evidence of effectiveness from trials. Generalisability across studies has been aided by the production of tools to describe key population attributes in standardised ways. Alongside traditional statistical methods, such as sub-group analyses and meta-regression, the use of standardised tools and the inclusion of qualitative research enable reviewers to synthesise a messy evidence base in ways that are useful and meaningful. Conclusions: Many reviews might be characterised as coming from a ‘convergent’ viewpoint; different (but similar) studies are brought together in order to identify a common measure of effect. Systematic reviews of social research are often ‘divergent’ in nature: heterogeneity is a given, and the requirement is for review methods to support reviews that make sense of this diversity while maintaining the core principles that underpin robust research synthesis.