Article type
Year
Abstract
Background:
Overviews of reviews should be fully and transparently reported to allow readers to assess the adequacy of the methods used and whether there are any threats to the validity of the overview findings.
Objectives:
To assess the completeness of reporting of methods in overviews.
Methods:
We used a published framework of methods for conducting an overview as the basis for assessing the adequacy of the reporting of the methods. A random sample of 50 overviews was identified through a search of MEDLINE and independently screened by two authors. One author assessed the completeness of reporting for all overviews, while a second author independently assessed 37 overviews.
Results:
A clinical rationale for undertaking the overview was reported in 29/50 and 26/50 defined the overviews' PICO elements (population, interventions, comparisons, outcomes). A full search strategy was reported in 18/50 overviews. Almost all overviews (49/50) searched bibliographic databases and 44/50 searched systematic review databases (e.g. Cochrane, Epistemonikos). Nearly half of the overviews (24/50) reported a process for selecting systematic reviews and for extracting data (23/50). Methods for assessing the risk of bias (RoB)/quality of the systematic reviews were reported in 38/50 overviews but were infrequently incorporated into the results and conclusions (14/38). Tools to assess RoB/quality of the systematic reviews included AMSTAR (23/50), OQAQ (9/50), R-AMSTAR (2/50) and ROBIS (1/50). Thirteen of 50 overviews reported how methods for RoB/quality of primary studies were assessed. No overviews reported how different tools to assess primary studies' RoB/quality were dealt with across systematic reviews. Few overviews (10/50) sufficiently detailed the methods used for narrative summary and one-quarter (12/50) reported conducting "narrative synthesis" without any further description of the method. However, fewer (3/50) reported methods of quantitative synthesis. Of the 17/50 overviews that reported assessing the certainty of the evidence (CoE), about half used GRADE (8/17), 5/17 used the Jadad tool and 2/17 used other tools to assess CoE.
Conclusions:
Results from this study provide a benchmark of the completeness of reporting of overview methods and indicate that improvement is needed. Development of a specific reporting guideline for overviews in combination with strategies to facilitate the uptake of the guideline are required.
Patient or healthcare consumer involvement:
None.
Overviews of reviews should be fully and transparently reported to allow readers to assess the adequacy of the methods used and whether there are any threats to the validity of the overview findings.
Objectives:
To assess the completeness of reporting of methods in overviews.
Methods:
We used a published framework of methods for conducting an overview as the basis for assessing the adequacy of the reporting of the methods. A random sample of 50 overviews was identified through a search of MEDLINE and independently screened by two authors. One author assessed the completeness of reporting for all overviews, while a second author independently assessed 37 overviews.
Results:
A clinical rationale for undertaking the overview was reported in 29/50 and 26/50 defined the overviews' PICO elements (population, interventions, comparisons, outcomes). A full search strategy was reported in 18/50 overviews. Almost all overviews (49/50) searched bibliographic databases and 44/50 searched systematic review databases (e.g. Cochrane, Epistemonikos). Nearly half of the overviews (24/50) reported a process for selecting systematic reviews and for extracting data (23/50). Methods for assessing the risk of bias (RoB)/quality of the systematic reviews were reported in 38/50 overviews but were infrequently incorporated into the results and conclusions (14/38). Tools to assess RoB/quality of the systematic reviews included AMSTAR (23/50), OQAQ (9/50), R-AMSTAR (2/50) and ROBIS (1/50). Thirteen of 50 overviews reported how methods for RoB/quality of primary studies were assessed. No overviews reported how different tools to assess primary studies' RoB/quality were dealt with across systematic reviews. Few overviews (10/50) sufficiently detailed the methods used for narrative summary and one-quarter (12/50) reported conducting "narrative synthesis" without any further description of the method. However, fewer (3/50) reported methods of quantitative synthesis. Of the 17/50 overviews that reported assessing the certainty of the evidence (CoE), about half used GRADE (8/17), 5/17 used the Jadad tool and 2/17 used other tools to assess CoE.
Conclusions:
Results from this study provide a benchmark of the completeness of reporting of overview methods and indicate that improvement is needed. Development of a specific reporting guideline for overviews in combination with strategies to facilitate the uptake of the guideline are required.
Patient or healthcare consumer involvement:
None.