Good news about bad news: Improvements in attention to adverse effects in Cochrane reviews (1997-2007) and future challenges

Tags: Oral
Bastian H, Schultheis A

Background: Knowledge about adverse effects (AEs) is essential to informed decision making. Assessments in earlier years revealed that many Cochrane reviews reviewed only effectiveness, thus failing to consider the potential for harm.

Objectives: To identify whether or not there was an increase in attention paid to adverse effects in Cochrane reviews between 1997 and 2007. A secondary objective was to identify whether reviews which had not considered adverse effects in 1997 had been updated.

Methods: Data gathered on all Cochrane reviews published by Issue 1 of 1997 (1) were reviewed (n=159 reviews). The 31 reviews (19%) which had not mentioned adverse effects were used as baseline data and their current status was reviewed by one author (HB). Data on outcome measures predetermined in the methods of all new Cochrane reviews in Issues 1 and 2 of the Cochrane Database of Systematic Reviews 5 and 10 years later (2002 and 2007) were collected. Both authors classified these as addressing or not addressing adverse effects. Differences were resolved by discussion. In addition, one author (AS) identified whether the abstracts mentioned findings on adverse effects. The interventions covered by the reviews were classified as: drugs (including herbal medicines), invasive interventions (including potentially psychologically invasive) or non-invasive interventions.

Results: In 1997, 19% of Cochrane reviews did not mention AEs. The number of reviews which did not specifically list AEs as predetermined outcome measures in 2002 was 42/143 (29%) and 28/188 (15%) in 2007. There was a significant drop in the number of reviews of drug interventions which did not list AEs as predetermined outcomes, from 18% in 1997 to 6% in 2007 (95% CI: 2.6%-21.5%). Many of the reviews (12/31) which did not mention AEs in 1997 are no longer on The Cochrane Library, having been withdrawn or converted into new reviews. Of the remaining 19 still online, only 4 now address AEs.

Conclusions: Cochrane reviews on drugs now pay more attention to adverse effects, although adverse effects remain unaddressed in many reviews. For some reviews, AEs are of far less importance than others. Future challenges include explaining why AEs are, legitimately, not always included as outcome measures. There needs to be more consideration of the need for improved methodology in updates of reviews.

(1) Bastian H, Middleton P. More good than harm: how well can Cochrane reviews help people decide? Cochrane Colloquium, Amsterdam, 1997.