Comparison of current and previous approaches to deal with publication bias in systematic reviews

Article type
Authors
Parekh-Bhurke S, Kwok C, Pang C, Hooper L, Loke Y, Ryder J, Sutton A, Hing C, Harvey I, Song F
Abstract
Background: Publication bias, if undetected, can lead to inaccurate estimates of treatment effect sizes and hence undermine the validity of clinical guidelines. It has been observed in the past that consequences of publication bias have been greatly ignored. However, a wider awareness of the detrimental effects of publication bias has promoted recent efforts to prevent and reduce publication bias. Objectives: To examine the approaches used to deal with publication bias in different types of systematic reviews published in 2006 and compare them to reviews published in 1996 and assessed previously. Methods: PubMed was searched for systematic reviews published in 2006 and randomly selected 100 RCT reviews of treatment effectiveness, 50 of diagnostic accuracy, 100 of risk factors, and 50 of gene-disease association. Independent data extraction of methods used to address publication bias was conducted by two reviewers. Results: A 21% increase was seen in the use of MEDLINE to identify relevant studies since 1996 followed by 31% increase in the use of references of identified studies. Use of The Cochrane Library increased from 5% to 58% and CINAHL from 8% to 24% in treatment reviews, 20% in diagnostic reviews and 18% is risk factor reviews in comparison to reviews published in 1996 (Figure 1). Non-English language studies were explicitly searched for in 39% of reviews published in 2006 compared to 19% in 1996. Efforts to explicitly search grey literature and/or unpublished studies increased to 61% from 35% in treatment reviews published in 1996. Twenty-six per cent of the reviews used funnel plots and related methods to test publication bias compared with less than 6% in 1996. Conclusions: There is a marked improvement in the measures taken to deal with publication bias within meta-analyses. However, few methods exist to deal with publication bias in the non-quantitative findings of systematic reviews.