Article type
Year
Abstract
Background: Narrative synthesis (NS) is commonly used in systematic reviews (SR), especially when there is a high level of complexity and heterogeneity. Yet developments to improve review methods have largely overlooked NS of quantitative data. Although NS guidance exists, it is rarely used and consensus is lacking about how quantitative data should be synthesised. Consequently, it is difficult to assess rigour and potential bias in NS.
Objectives: To assess the methods and adequacy of reporting of NS of quantitative data in SRs.
Methods: Focussing on SRs of public health interventions, we used a random 20% (n = 474) sample of SRs from the McMaster Health Evidence database (2010 onward) to identify SRs using NS. Informed by key sources on NS methods, we extracted data from 29% (n=72) of reviews using NS on: SR characteristics, justification for NS, management of conceptual and methodological heterogeneity including clarity of groupings used in the NS, links between data and text, and adequacy of NS description.
Results: In total, 48% of reviews (n = 215) used NS only and 44% (n = 195) used meta-analysis only; 8% of reviews (n = 36) used NS and meta-analysis. Of the reviews using NS, 75% included non-randomised studies, and 23% (n = 58/251) referenced a protocol. Description or justification for use of NS was limited and often absent. Investigation and management of heterogeneity was unclear, and data were not presented transparently so as to facilitate links to the synthesis findings.
Conclusions: Despite frequent use of NS for quantitative data, lack of transparency in reporting makes it difficult to assess the rigour and reliability of SR findings. Failure to manage heterogeneity and justify groupings used in the synthesis further prevents assessment of the appropriateness and usefulness of the synthesis. We estimate that NS is used in > 30% of all SRs. The lack of transparency raises concern about the potential for bias in a large volume of the SR evidence base, and is a potential threat to evidence-informed decision-making. There is an urgent need for a programme of methodological development to underpin and improve NS of quantitative data.
Objectives: To assess the methods and adequacy of reporting of NS of quantitative data in SRs.
Methods: Focussing on SRs of public health interventions, we used a random 20% (n = 474) sample of SRs from the McMaster Health Evidence database (2010 onward) to identify SRs using NS. Informed by key sources on NS methods, we extracted data from 29% (n=72) of reviews using NS on: SR characteristics, justification for NS, management of conceptual and methodological heterogeneity including clarity of groupings used in the NS, links between data and text, and adequacy of NS description.
Results: In total, 48% of reviews (n = 215) used NS only and 44% (n = 195) used meta-analysis only; 8% of reviews (n = 36) used NS and meta-analysis. Of the reviews using NS, 75% included non-randomised studies, and 23% (n = 58/251) referenced a protocol. Description or justification for use of NS was limited and often absent. Investigation and management of heterogeneity was unclear, and data were not presented transparently so as to facilitate links to the synthesis findings.
Conclusions: Despite frequent use of NS for quantitative data, lack of transparency in reporting makes it difficult to assess the rigour and reliability of SR findings. Failure to manage heterogeneity and justify groupings used in the synthesis further prevents assessment of the appropriateness and usefulness of the synthesis. We estimate that NS is used in > 30% of all SRs. The lack of transparency raises concern about the potential for bias in a large volume of the SR evidence base, and is a potential threat to evidence-informed decision-making. There is an urgent need for a programme of methodological development to underpin and improve NS of quantitative data.