Systematic methods for identifying evidence for broad review questions: looking beyond titles and abstracts

Article type
Authors
O’Mara-Eves AJ1, Brunton G1, Thomas J1, Kavanagh J1, Oliver S1
1EPPI Centre, Institute of Education, UK
Abstract
Background: Systematic reviews (SRs) of broad topics, especially in public health, can cover many discipline areas. Exhaustive searching for such widely dispersed evidence can be costly, and different terminology used across disciplines hampers identification. A further complication arises when the review question’s focus is peripheral to the aims of the primary research, and therefore might not be referred to in the titles and abstracts of the primary studies. Such evidence is likely to be missed by conventional review methods, but there are no clear guidelines for systematic alternatives.

Objectives: To demonstrate alternative systematic ways of identifying relevant evidence where the key concepts are of interest across various disciplines and are generally not focal to the primary studies’ aims. Specifically, we sought to identify evidence on community engagement in public health interventions to reduce health inequalities.

Methods: We conducted a systematic search for SRs on public health interventions, supplemented by searches of databases of trials. We capitalised on the systematic searches used in those SRs by identifying primary studies within the reviews that were relevant to our review. Importantly, we used information in the SR evidence tables and full-text document to gather more information about the included studies than was evident in the primary studies’ own titles or abstracts. The initial search strategy also used text mining to identify synonyms for the concept ‘community engagement’, to ensure that cross-disciplinary terms were detected.

Results: We identified over 400 primary studies cited in SRs after full-text screening—about ten times more than expected. We estimate that about one third of the studies that were identified would have been missed by conventional methods.

Conclusions: The identification strategy was effective and is useful for reviews of broad research questions or where the key concepts are unlikely to be the main focus of primary research.