Reaching certainty: a descriptive study of ‘stable’ Cochrane Reviews and coming to firm conclusions

Article type
Authors
Bastian H1, Hemkens L2
1National Center for Biotechnology Information, National Library of Medicine, NIH, USA
2Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland
Abstract
Background: According to the Cochrane Handbook, a ‘stable’ review is ‘one that is highly likely to maintain its current relevance for the foreseeable future’.

Objectives: To monitor the extent and reasons for Cochrane Reviews (CR) being designated stable, and to describe the reviews designated by the CR authors as having a conclusion that would be unlikely to change with new trials (called here a ‘firm conclusion’).

Methods: Stable reviews were identified in the February 2013 Issue of the Cochrane Database of Systematic Reviews (CDSR). Data on the year and reasons of the designation ‘stable’ were extracted from the ‘What’s new’ section. When reasons were not clearly reported there, the abstract, discussion and conclusions sections were searched. Both authors agreed on categories. The conclusions and body of evidence of reviews with firm conclusions were described.

Results: There were 180 stable reviews among 5137 ‘non-withdrawn’ intervention and diagnostic CRs in the CDSR (3.5%). The number of reviews being designated stable has been increasing in recent years. Reasons for the designation were often not explained in the ‘What’s new’ section, and a total of 26 reviews gave no clear reason in the review for the categorizing reviews as stable. Where a reason was given (154 reviews), the most common category was a belief that future trials were unlikely (36%) (see Table 1). Only 16 reviews are designated as having firm conclusions, mostly because there was no evidence of benefit, or any benefits were either clinically unimportant or not sustained (11/16 reviews, 69%).

Conclusions: Reasons for the decision to designate a review ‘stable’ are often poorly reported. Cochrane authors rarely conclude that new trials would be unlikely to change the conclusions of their reviews. They may be more likely to do so when there is an absence of evidence of effect.