Article type
Year
Abstract
Background: It is the policy of the Cochrane Collaboration that reviews should either be updated within two years or should have a commentary added to explain why this is done less frequently [1]. Currently there is no evidence about the best method of assessing the need to update a Cochrane review. Spending time updating could waste valuable resources and introduce bias related to the slower publication of studies with negative and inconclusive results [2].
Objectives: 1. To determine the proportion of reviews where updating resulted in a change to the conclusions of the review [3]. 2. To determine whether an updated review results in a more "confident" conclusion.
Methods: Using the Cochrane Database of Systematic Reviews (CDSR) we identified a cohort of reviews that had been updated with additional included studies between issue 2, 1998 and issue 2, 2002 [4]. From this cohort we excluded any reviews that had additional studies as a result of changes to the review question. The remaining reviews were divided into two cohorts: those that changed their conclusions and those that did not. For the reviews that did not change their conclusion, a primary outcome was identified using a pre-specified rule [2]. We calculated the difference between the precision of the primary outcome of each review pair.
Results: A total of 137 (36%) reviews added new trials. Thirty (22%) of these reviews addressed a different research question (different patient group and/or intervention) and were excluded from this analysis. This left 107 (78%) reviews that addressed the same research question and included additional trials. Of this group, 14 (13%) reviews had a change in the conclusion of the review following the update and 93 (87%) had no change in conclusion. Results will be presented on what proportion of these reviews with no change in conclusion had a more "confident" conclusion.
Conclusions: The updating of Cochrane reviews consumes considerable time and resources and in many cases may not change the conclusion or lead to a more confident conclusion. A method for identifying reviews where updates are more likely to result in a change of conclusion or increased confidence in a conclusion will assist review groups in prioritising reviews for updates.
Acknowledgements: Jason Wasiak for his work on the preliminary part of this project and Jon Deeks for methodological advice.
References: 1. Alderson P, Green S, Higgins JPT, editors. Cochrane Reviewers' Handbook 4.2.1 [updated December 2003]. In: The Cochrane Library, Issue 1, 2004. Chichester, UK: John Wiley & Sons, Ltd. 2. Higgins J. How should we interpret updated meta-analyses? Proceedings of the 7th Cochrane Colloquium, Rome, October 1999. 3. Silagy CA, Middleton P & Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287(21):2831-4 4. McDonald S, Green S, editors. What constitutes an update of a systematic review?: a survey of the first 377 reviews. Proceedings of the 11th Cochrane Colloquium; 2003 Oct 26-31; Barcelona, Spain.
Objectives: 1. To determine the proportion of reviews where updating resulted in a change to the conclusions of the review [3]. 2. To determine whether an updated review results in a more "confident" conclusion.
Methods: Using the Cochrane Database of Systematic Reviews (CDSR) we identified a cohort of reviews that had been updated with additional included studies between issue 2, 1998 and issue 2, 2002 [4]. From this cohort we excluded any reviews that had additional studies as a result of changes to the review question. The remaining reviews were divided into two cohorts: those that changed their conclusions and those that did not. For the reviews that did not change their conclusion, a primary outcome was identified using a pre-specified rule [2]. We calculated the difference between the precision of the primary outcome of each review pair.
Results: A total of 137 (36%) reviews added new trials. Thirty (22%) of these reviews addressed a different research question (different patient group and/or intervention) and were excluded from this analysis. This left 107 (78%) reviews that addressed the same research question and included additional trials. Of this group, 14 (13%) reviews had a change in the conclusion of the review following the update and 93 (87%) had no change in conclusion. Results will be presented on what proportion of these reviews with no change in conclusion had a more "confident" conclusion.
Conclusions: The updating of Cochrane reviews consumes considerable time and resources and in many cases may not change the conclusion or lead to a more confident conclusion. A method for identifying reviews where updates are more likely to result in a change of conclusion or increased confidence in a conclusion will assist review groups in prioritising reviews for updates.
Acknowledgements: Jason Wasiak for his work on the preliminary part of this project and Jon Deeks for methodological advice.
References: 1. Alderson P, Green S, Higgins JPT, editors. Cochrane Reviewers' Handbook 4.2.1 [updated December 2003]. In: The Cochrane Library, Issue 1, 2004. Chichester, UK: John Wiley & Sons, Ltd. 2. Higgins J. How should we interpret updated meta-analyses? Proceedings of the 7th Cochrane Colloquium, Rome, October 1999. 3. Silagy CA, Middleton P & Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287(21):2831-4 4. McDonald S, Green S, editors. What constitutes an update of a systematic review?: a survey of the first 377 reviews. Proceedings of the 11th Cochrane Colloquium; 2003 Oct 26-31; Barcelona, Spain.