Article type
Year
Abstract
Background:
Overviews (of reviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overview authors should report the methodological quality of the included SRs; this can be done using the Assessment of Multiple Systematic Reviews (AMSTAR) tool.
Objectives:
We examined methodological considerations involved in using AMSTAR in overviews. We assessed: 1) what differences exist in AMSTAR scores across Cochrane and non-Cochrane reviews, 2) challenges involved in using AMSTAR to assess quality of included SRs, and 3) feasibility of using the AMSTAR score as an inclusion criterion.
Methods:
We selected a sample of seven overviews in different clinical areas and searched for all SRs meeting each overview's inclusion criteria. Two reviewers conducted AMSTAR assessments for each SR and resolved discrepancies through consensus. Challenges and issues surrounding the use of the AMSTAR tool were noted and discussed.
Results:
To date, we have assessed 72 SRs in four clinical areas (acute otitis media, bronchiolitis, eczema, procedural sedation). Average AMSTAR scores (/11) were much higher for Cochrane (9.95 ± 1.08) compared with non-Cochrane (4.95 ± 2.25) SRs (Table 1). Average agreement between reviewers was high overall, but slightly higher for Cochrane (90.9%) compared with non-Cochrane (85.1%) SRs. AMSTAR assessments were difficult when SRs were broader in scope than the overview's clinical question and when SRs examining similar interventions made different methodological decisions or differed in conclusions.
Conclusions:
The relatively high agreement between reviewers when using AMSTAR for Cochrane and non-Cochrane SRs suggests that AMSTAR can be used successfully in the context of overviews. Cochrane SRs are often higher quality than non-Cochrane SRs; this seems to be largely driven by reporting and may not be associated with methodological quality. Nevertheless, non-Cochrane SRs with low AMSTAR scores may be missing important information and may be challenging to use in overviews. A minimum AMSTAR score may be useful as an inclusion criterion for overviews.
Overviews (of reviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overview authors should report the methodological quality of the included SRs; this can be done using the Assessment of Multiple Systematic Reviews (AMSTAR) tool.
Objectives:
We examined methodological considerations involved in using AMSTAR in overviews. We assessed: 1) what differences exist in AMSTAR scores across Cochrane and non-Cochrane reviews, 2) challenges involved in using AMSTAR to assess quality of included SRs, and 3) feasibility of using the AMSTAR score as an inclusion criterion.
Methods:
We selected a sample of seven overviews in different clinical areas and searched for all SRs meeting each overview's inclusion criteria. Two reviewers conducted AMSTAR assessments for each SR and resolved discrepancies through consensus. Challenges and issues surrounding the use of the AMSTAR tool were noted and discussed.
Results:
To date, we have assessed 72 SRs in four clinical areas (acute otitis media, bronchiolitis, eczema, procedural sedation). Average AMSTAR scores (/11) were much higher for Cochrane (9.95 ± 1.08) compared with non-Cochrane (4.95 ± 2.25) SRs (Table 1). Average agreement between reviewers was high overall, but slightly higher for Cochrane (90.9%) compared with non-Cochrane (85.1%) SRs. AMSTAR assessments were difficult when SRs were broader in scope than the overview's clinical question and when SRs examining similar interventions made different methodological decisions or differed in conclusions.
Conclusions:
The relatively high agreement between reviewers when using AMSTAR for Cochrane and non-Cochrane SRs suggests that AMSTAR can be used successfully in the context of overviews. Cochrane SRs are often higher quality than non-Cochrane SRs; this seems to be largely driven by reporting and may not be associated with methodological quality. Nevertheless, non-Cochrane SRs with low AMSTAR scores may be missing important information and may be challenging to use in overviews. A minimum AMSTAR score may be useful as an inclusion criterion for overviews.