Article type
Year
Abstract
Background: Despite the Collaboration’s policy on updating, many Cochrane reviews are not up to date. Updating can be as resource intensive as producing the original review and a priority-based approach may be more efficient than an arbitrary rigid time-based approach.
Objectives: To develop a prediction tool for assessing the need to update a review and for ranking a portfolio of reviews being considered for updating.
Methods: We identified 'signals’ through the literature together with novel ideas. We hypothesised these could potentially change the conclusions of a review and included the availability of a new trial, sample sizes of new trials etc. Further signals were estimated through simulation, e.g. the estimated power of the new evidence to change conclusions of a meta-analysis. Only information on the sample sizes of each new trial was required to estimate all signals. To develop an equation predicting the probability a review’s conclusions would change based on the signals, a sample of Cochrane reviews flagged as updated during 2009 were identified. The predictive ability of the signals was evaluated using logistic regression; the outcome was whether conclusions of a review changed or not. The 'best’ prediction equation was validated using Jackknife cross-validation methods.
Results: Thirteen signals were identified of which five were stochastic (Figure1). We found 75 reviews where at least one new study had been added to the meta-analysis of the outcome on which the conclusions of the review was based. The signals and prediction tool will be presented in detail at the Colloquium.
Conclusions: The prediction tool provides a quantitative assessment of the likelihood that new trials will overturn conclusions of a review. Other factors may influence the decision to update but using the tool could ensure that reviews most sensitive to change due to new data are regularly updated.
Objectives: To develop a prediction tool for assessing the need to update a review and for ranking a portfolio of reviews being considered for updating.
Methods: We identified 'signals’ through the literature together with novel ideas. We hypothesised these could potentially change the conclusions of a review and included the availability of a new trial, sample sizes of new trials etc. Further signals were estimated through simulation, e.g. the estimated power of the new evidence to change conclusions of a meta-analysis. Only information on the sample sizes of each new trial was required to estimate all signals. To develop an equation predicting the probability a review’s conclusions would change based on the signals, a sample of Cochrane reviews flagged as updated during 2009 were identified. The predictive ability of the signals was evaluated using logistic regression; the outcome was whether conclusions of a review changed or not. The 'best’ prediction equation was validated using Jackknife cross-validation methods.
Results: Thirteen signals were identified of which five were stochastic (Figure1). We found 75 reviews where at least one new study had been added to the meta-analysis of the outcome on which the conclusions of the review was based. The signals and prediction tool will be presented in detail at the Colloquium.
Conclusions: The prediction tool provides a quantitative assessment of the likelihood that new trials will overturn conclusions of a review. Other factors may influence the decision to update but using the tool could ensure that reviews most sensitive to change due to new data are regularly updated.
Images
