Article type
Year
Abstract
Background:
Proliferation of systematic reviews (SRs) and escalating demand for accessible evidence have driven interest in overviews of reviews. Unique challenges arise in synthesising review evidence, for which guidance is yet to be agreed. These challenges have ramifications for assessing the certainty of evidence using GRADE or other approaches. Authors must decide, for example, how to 1) account for limitations of the included SRs, 2) deal with overlapping data (e.g. where the same primary studies contribute to meta-analyses in multiple reviews), and 3) handle missing or discordant data needed to assess certainty (e.g. conflicting risk of bias assessments).
Objectives:
To inform guidance for assessing the certainty of evidence in overviews by examining current practice and expert perspectives.
Methods:
We undertook a systematic review of methods used to assess the certainty of the evidence in all eligible overviews (i.e. those reporting formal or informal assessment of certainty) indexed in MEDLINE (January 2015 to March 2017). We carried out interviews followed by a workshop (Global Evidence Summit, 2017) with GRADE, overview and guideline methodologists about their views on assessing certainty of evidence in overviews. Methodologists were presented with scenarios common in overviews (e.g. two overlapping meta-analyses from SRs of different quality and currency), and asked to describe and comment on approaches to rating certainty.
Results:
From 1707 citations, 41 of 131 overviews were included; 19 extracted or reported GRADE assessments (most without modification for overviews), 16 used another tool and six used ad hoc criteria. Participants in interviews and the workshop agreed that both certainty of the primary evidence and the SR process are important, but held diverse views on whether to integrate SR limitations in the rating of certainty. The overview purpose and methods dictate approaches to assessing certainty if data are overlapping, missing or discordant. For example, in dealing with overlap, the approach differs if selecting one meta-analysis to GRADE rather than conducting a new meta-analysis.
Conclusions:
Our results provide principles for assessing the certainty of evidence in an overview. However, different approaches may be warranted for incorporating limitations of the SR when assessing certainty of the evidence.
Patient or healthcare consumer involvement:
None.
Proliferation of systematic reviews (SRs) and escalating demand for accessible evidence have driven interest in overviews of reviews. Unique challenges arise in synthesising review evidence, for which guidance is yet to be agreed. These challenges have ramifications for assessing the certainty of evidence using GRADE or other approaches. Authors must decide, for example, how to 1) account for limitations of the included SRs, 2) deal with overlapping data (e.g. where the same primary studies contribute to meta-analyses in multiple reviews), and 3) handle missing or discordant data needed to assess certainty (e.g. conflicting risk of bias assessments).
Objectives:
To inform guidance for assessing the certainty of evidence in overviews by examining current practice and expert perspectives.
Methods:
We undertook a systematic review of methods used to assess the certainty of the evidence in all eligible overviews (i.e. those reporting formal or informal assessment of certainty) indexed in MEDLINE (January 2015 to March 2017). We carried out interviews followed by a workshop (Global Evidence Summit, 2017) with GRADE, overview and guideline methodologists about their views on assessing certainty of evidence in overviews. Methodologists were presented with scenarios common in overviews (e.g. two overlapping meta-analyses from SRs of different quality and currency), and asked to describe and comment on approaches to rating certainty.
Results:
From 1707 citations, 41 of 131 overviews were included; 19 extracted or reported GRADE assessments (most without modification for overviews), 16 used another tool and six used ad hoc criteria. Participants in interviews and the workshop agreed that both certainty of the primary evidence and the SR process are important, but held diverse views on whether to integrate SR limitations in the rating of certainty. The overview purpose and methods dictate approaches to assessing certainty if data are overlapping, missing or discordant. For example, in dealing with overlap, the approach differs if selecting one meta-analysis to GRADE rather than conducting a new meta-analysis.
Conclusions:
Our results provide principles for assessing the certainty of evidence in an overview. However, different approaches may be warranted for incorporating limitations of the SR when assessing certainty of the evidence.
Patient or healthcare consumer involvement:
None.