Article type
Abstract
Background: Overviews of the findings of systematic reviews of health systems interventions can help policy makers and other stakeholders to identify strategies for strengthening health systems. Methods for conducting such overviews are still evolving.
Objectives: 1) To examine the strengths and weaknesses of the methods used for four overviews of reviews of health systems reviews relevant to low income countries; and. 2) To discuss the methodological lessons from undertaking these overviews.
Methods: The overview methods were as follows: we searched PDQ Evidence – a database of evidence for health systems decisions – for relevant reviews and included well-conducted reviews published between 2005 and 2016 of studies that assessed the effects of governance, financial and delivery arrangements and implementation strategies. We excluded reviews that had limitations that were important enough that the findings of the review were not reliable. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, summary of findings tables and assessments of relevance of findings to low income countries. We then reflected on the strengths and weaknesses of the methods used and identified lessons learned.
Results: The strengths of our methods included: 1) a ‘layered’ approach to evidence presentation; 2) multiple checks through the review process for the reliability and applicability of reviews; and, 3) use of a well-tested format (SUPPORT summaries) for presenting review findings. Key limitations: 1) the process was very time and resource-intensive; and, 2) the preparation of summaries is a highly specialised task. Challenges included multiple, overlapping reviews on many topics; few data on resource use, adverse effects and equity impacts in reviews; and the wider challenge of summarising and presenting complex information on health systems.
Conclusions:Producing overviews would be facilitated by the wider use of summaries of findings tables and user-friendly formats in reviews of health systems interventions.
Objectives: 1) To examine the strengths and weaknesses of the methods used for four overviews of reviews of health systems reviews relevant to low income countries; and. 2) To discuss the methodological lessons from undertaking these overviews.
Methods: The overview methods were as follows: we searched PDQ Evidence – a database of evidence for health systems decisions – for relevant reviews and included well-conducted reviews published between 2005 and 2016 of studies that assessed the effects of governance, financial and delivery arrangements and implementation strategies. We excluded reviews that had limitations that were important enough that the findings of the review were not reliable. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, summary of findings tables and assessments of relevance of findings to low income countries. We then reflected on the strengths and weaknesses of the methods used and identified lessons learned.
Results: The strengths of our methods included: 1) a ‘layered’ approach to evidence presentation; 2) multiple checks through the review process for the reliability and applicability of reviews; and, 3) use of a well-tested format (SUPPORT summaries) for presenting review findings. Key limitations: 1) the process was very time and resource-intensive; and, 2) the preparation of summaries is a highly specialised task. Challenges included multiple, overlapping reviews on many topics; few data on resource use, adverse effects and equity impacts in reviews; and the wider challenge of summarising and presenting complex information on health systems.
Conclusions:Producing overviews would be facilitated by the wider use of summaries of findings tables and user-friendly formats in reviews of health systems interventions.