Article type
Year
Abstract
Background: While a systematic review is one of the logical and scientific methods used frequently in health technology assessment, it is subjected to reporting bias as it fundamentally uses primary studies. Among reporting biases, publication bias might lead to overestimation of effect size in systematic reviews with meta-analyses, which could affect the credibility of study results.
Objectives: In this study, the current status of meta-analysis studies reporting small study effect and publication bias was investigated, and the biases were explored and adjusted to examine how the effect estimates in meta-analyses could change.
Methods: Systematic reviews with meta-analyses published in some major journals within recent 10 years were selected according to inclusion/exclusion criteria. Contour-enhanced funnel plot, Egger’s test, and Begg’s test were applied to explore small study effect and publication bias, and Egger’s variance regression model was adopted for adjustment.
Results: Among 170 studies selected, 82 (48%) reported publication bias. Most of the studies used funnel plot alone or combined with other statistical methods, and some applied only statistical models including Egger’s test and/or Begg’s test. For exploration and adjustment of publication bias, 17 meta-analyses of continuous data without between-study-heterogeneity were selected. They did not explore small study effect and publication bias or reported low possibility. However, possibility of small study effect appeared in nine (53%), probably resulting from publication bias in eight of nine. The results of meta-analyses, significant effectiveness of interventions were not robust after adjustment in four (24%) meta-analyses.
Conclusions: Small study effect and publication bias have not intensively investigated in the studies published even in some high profile major journals and there were indications for possible alterations of conclusions. More attention to exploring such a bias should be paid when reporting results of systematic reviews.
Objectives: In this study, the current status of meta-analysis studies reporting small study effect and publication bias was investigated, and the biases were explored and adjusted to examine how the effect estimates in meta-analyses could change.
Methods: Systematic reviews with meta-analyses published in some major journals within recent 10 years were selected according to inclusion/exclusion criteria. Contour-enhanced funnel plot, Egger’s test, and Begg’s test were applied to explore small study effect and publication bias, and Egger’s variance regression model was adopted for adjustment.
Results: Among 170 studies selected, 82 (48%) reported publication bias. Most of the studies used funnel plot alone or combined with other statistical methods, and some applied only statistical models including Egger’s test and/or Begg’s test. For exploration and adjustment of publication bias, 17 meta-analyses of continuous data without between-study-heterogeneity were selected. They did not explore small study effect and publication bias or reported low possibility. However, possibility of small study effect appeared in nine (53%), probably resulting from publication bias in eight of nine. The results of meta-analyses, significant effectiveness of interventions were not robust after adjustment in four (24%) meta-analyses.
Conclusions: Small study effect and publication bias have not intensively investigated in the studies published even in some high profile major journals and there were indications for possible alterations of conclusions. More attention to exploring such a bias should be paid when reporting results of systematic reviews.