Evaluating two statistical methods to predict when a meta-analysis will gain sufficient power to change the conclusions of a Cochrane review

Article type
Authors
Sutton A, Gamble C, Donegan S, Garner P
Abstract
Background: Updating Cochrane reviews takes time. Effective tools to help prioritise which reviews are in most urgent need of updating would be helpful.

Objectives: To assess retrospectively the performance of two statistical methods for identifying when enough new evidence is available potentially to overturn the indecisive results of a primary outcome within a Cochrane systematic review. This work is done with the view of using such methods prospectively in the future to inform which Cochrane reviews are in most urgent need of an update.

Methods: We adapted two methods to predict the sample size of a trial required to change the findings of a meta-analysis to predict whether trials awaiting assessment would change the conclusions of a Cochrane review, and applied this to reviews in the Cochrane Infectious Diseases Group. Reviews reporting a binary endpoint and accumulating evidence over a four or more year time period were potentially eligible for inclusion. Studies published within three years of the most recent study in a review were excluded and a meta-analysis of the primary outcome re-calculated. If this pooled estimate was non-significant then this review was included in our study. Only knowing the numbers of persons randomized in the excluded studies, the methods were applied to estimate the power of including the extra studies and hence the likelihood that the conclusion of the systematic review would change when updated. These predictions were compared with what actually happened when all studies were included for each review.

Results: The accuracy of power predictions of the updated meta-analyses using both methods will be compared to reality.

Conclusions: This research evaluates whether a quick assessment of trials awaiting assessment can be used to predict whether they will alter the findings of the Cochrane review, and hence allow review teams to prioritise reviews for updating.