Article type
Year
Abstract
Background: Systematic reviews combine the quantitative evidence from primary studies using statistical methods. The correct choice, implementation and interpretation of these methods are key to obtain reliable estimates of effect sizes. Although the Cochrane Handbook offers advice about thesemethods, their implementation is not easy and in many cases requires statistical support.
Objective: To assess the use of statistical methods in systematic reviews published in the Cochrane Library.
Methods: All the new reviews published in the Cochrane Library issue 2, 2013—with at least one meta-analysis—were selected for appraisal. Two evaluators assessed independently each selected review using items 9 and 10 of the AMSTAR tool.
Results: Fourteen of the 35 new systematic reviews retrieved were selected for appraisal. They included a median of 5 studies (2–39) and presented a median of 9.5 forest plots (1–82). All of them stated in the Methods section how they would assess heterogeneity. The most commonly planned methods were the I2 statistic and the chi-squared test. Eleven of the 14 reviews stated in the Methods section how they would examine the causes of heterogeneity. All of them planned to use either subgroup or sensitivity analyses. In the Results section, potentially moderate/large heterogeneity was identified in 9 of the 14 reviews. Five of these nine reviews did not explore potential causes of heterogeneity. Nine of the 14 reviews stated in their Methods section how they would assess publication bias, but only four described an assessment in their Results or Discussion section.
Conclusions: There are deficiencies in the use of statistical methods for measuring and investigating heterogeneity and publication bias in Cochrane Reviews. Because of the limited number of reviews assessed, our findings should be considered as preliminary, and further work is needed for identifying factors associated with the deficiencies identified.
Objective: To assess the use of statistical methods in systematic reviews published in the Cochrane Library.
Methods: All the new reviews published in the Cochrane Library issue 2, 2013—with at least one meta-analysis—were selected for appraisal. Two evaluators assessed independently each selected review using items 9 and 10 of the AMSTAR tool.
Results: Fourteen of the 35 new systematic reviews retrieved were selected for appraisal. They included a median of 5 studies (2–39) and presented a median of 9.5 forest plots (1–82). All of them stated in the Methods section how they would assess heterogeneity. The most commonly planned methods were the I2 statistic and the chi-squared test. Eleven of the 14 reviews stated in the Methods section how they would examine the causes of heterogeneity. All of them planned to use either subgroup or sensitivity analyses. In the Results section, potentially moderate/large heterogeneity was identified in 9 of the 14 reviews. Five of these nine reviews did not explore potential causes of heterogeneity. Nine of the 14 reviews stated in their Methods section how they would assess publication bias, but only four described an assessment in their Results or Discussion section.
Conclusions: There are deficiencies in the use of statistical methods for measuring and investigating heterogeneity and publication bias in Cochrane Reviews. Because of the limited number of reviews assessed, our findings should be considered as preliminary, and further work is needed for identifying factors associated with the deficiencies identified.