Background: There is strong evidence of publication bias, whereby the results of positive trials are more likely to be published than negative or inconclusive ones. Considerable time and effort is needed to locate and obtain data from unpublished trials. However excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.

Objectives: To explore and quantify the impact of trials either unpublished, or published only as abstracts, on the results of all completed individual patient data (IPD) meta-analyses coordinated by our group (5 meta-analysis projects, 12 therapeutic comparisons).

Methods: For each of the comparisons within the IPD meta-analysis projects, results were calculated for (A) fully published trials only, (B) published trials + abstracts, and (C) published trials + abstracts + unpublished trials. All analyses were calculated from IPD. There is therefore no difference in type of data available between trials and no confounding factors in terms of excluded patients or duration of follow up.

Results: [Table excluded]

Discussion: Analyses based only on trials that were fully published at the time the IPD meta-analyses were conducted, tend to give more favourable results. The addition of data from trials presented only as abstracts or those that were unpublished give less encouraging results and in most cases move the estimated treatment effect towards a null result. In the PORT meta-analysis the less favourable, unpublished results move the result further away from a null effect and increase the estimated harm of treatment. Although the differences in effect size are relatively modest in these examples, the trend is worrying. All systematic reviews should at least identify unpublished trials and where possible, obtain data from them. [table excluded]