Article type
Year
Abstract
Background: Systematic review, followed by meta-analysis, is one of the most powerful and widespread tools for understanding efficacy and safety evaluation of medical interventions. However, statistical heterogeneity is one of the most serious issues when conducting this type of analysis. Cochran Q coefficient and I2 (I-squared) are the main statistics used to evaluate it. Confidence intervals (CI) can be calculated for both of them in order to determine their precision.
Objectives: The aim of this study was to review the application of the method to determine statistical heterogeneity, and its precision, in a group of selected systematic reviews that also considered meta-analysis and are published by Cochrane.
Methods: We searched in PubMed and Wiley for systematic reviews with meta-analysis that investigated chronic obstructive pulmonary disease and were produced by the Airways Group and published on the Cochrane Library. We included systematic reviews published between January 2013 and February 2015. We extracted the number of studies used in subgroup and total analysis, the Q coefficient, its P value, I2 statistics and their CI. If the CI was not present, we calculated it using the method proposed by Higgins et al.
Results: We included five reviews in the analysis. None reported the CI of the I2. The calculated CIs were consistently large, showing heterogeneity that could be interpreted as mild to severe. We found inconsistencies between the interpretation of the heterogeneity of the studies and the calculated CI in all of them, especially when the authors stated that there was no heterogeneity.
Conclusions: Heterogeneity is a serious concern in meta-analysis. We found inconsistencies between the interpretation of the reported values and the CIs of the I2. We propose that authors and also reviewers consider the existence of heterogeneity as an issue that really affects meta-analysis results because the selection of whether to apply a random-effects or fixed-effect model depends on its presence.
Objectives: The aim of this study was to review the application of the method to determine statistical heterogeneity, and its precision, in a group of selected systematic reviews that also considered meta-analysis and are published by Cochrane.
Methods: We searched in PubMed and Wiley for systematic reviews with meta-analysis that investigated chronic obstructive pulmonary disease and were produced by the Airways Group and published on the Cochrane Library. We included systematic reviews published between January 2013 and February 2015. We extracted the number of studies used in subgroup and total analysis, the Q coefficient, its P value, I2 statistics and their CI. If the CI was not present, we calculated it using the method proposed by Higgins et al.
Results: We included five reviews in the analysis. None reported the CI of the I2. The calculated CIs were consistently large, showing heterogeneity that could be interpreted as mild to severe. We found inconsistencies between the interpretation of the heterogeneity of the studies and the calculated CI in all of them, especially when the authors stated that there was no heterogeneity.
Conclusions: Heterogeneity is a serious concern in meta-analysis. We found inconsistencies between the interpretation of the reported values and the CIs of the I2. We propose that authors and also reviewers consider the existence of heterogeneity as an issue that really affects meta-analysis results because the selection of whether to apply a random-effects or fixed-effect model depends on its presence.