Sponsorship, bias and methodology: Cochrane reviews compared with paper-based meta-analyses of the same drugs

Jorgensen A, Gotzsche P

Background: Reviews show that bias in published studies sponsored by the industry is common and that the results and conclusions tend to favour the sponsor's product [1-3]. It would therefore be expected that systematic reviews and meta-analyses sponsored by the pharmaceutical industry are also biased.

Objectives: To compare Cochrane reviews of drugs with industry supported meta-analyses of the same drugs in the same diseases regarding methodological quality, estimated effect and conclusion.

Methods: The primary aim was to identify 30 sets of meta-analyses consisting of one Cochrane review and one industry supported paper-based meta-analysis. The meta-analyses of a set must have had compared the same two drugs in the same disease, with the same type of main outcome, must have been published within +-2 years from the year of most recent substantive amendment of the Cochrane Review, and must be published in full. The Cochrane Library, issue 1, 2003, was searched for reviews with meta-analyses comparing two drugs. For every index Cochrane review, the drugs and the disease were combined in a systematic search for industry supported paper-based meta-analyses in MEDLINE (January 1966 to July 2003) and EMBASE (1980 to August 2003). Data extraction and quality assessment were done independently by two reviewers, partly by using an index to evaluate scientific quality of the reviews [4,5] and a scale to grade the conclusions [3].

Results: 178 of 1596 Cochrane reviews had a least one meta-analysis comparing two drugs. The MEDLINE/EMBASE search resulted in paper-based meta-analyses for 73 of the 178 Cochrane reviews. We excluded 66 paper-based meta-analyses because they were published beyond +-2 years (29), did not contain a proper meta-analysis (7), specified support from non-industry sources (9), had authors in common with the Cochrane review (2), were not on the effect of drugs (3), were inaccessible (2), or had unclear support (14). Thus, 7 sets were included. The results of these 7 sets as well as the results for the 15 sets with unclear sponsorship of the paper-based reviews will be presented at the colloquium.

Conclusion: Will be presented at the colloquium.

References: 1. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsored and research outcome and quality: systematic review. BMJ. 2003;326:1167-70. 2. Melander H, Ahlquist-Rastad J, Meiser G, Beermann B. Evidence b(i)ased medicine - selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326:1171-3. 3. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290:921-8. 4. Oxman AD, Guyatt GH. Guidelines for reading literature reviews. Can Med Assoc J. 1988;138:697-703. 5. Jadad A, Cook D, JA, KT, Tugwell P, Moher M. Methodology and Reports of Systematic Reviews and Meta-analysis: A Comparison of Cochrane Reviews with Articles Published in Paper-Based Journals. JAMA. 1998;280:278-80.