Updating Cochrane reviews: vigilance required?

Article type
Authors
Ward K, Taylor F, Bickerdike E, Ebrahim S
Abstract
Background: With increased importance placed on updating systematic reviews, the question of thoroughly checking original data arises when new authors take on the update. During the current update of a large systematic review within the Cochrane Heart Group, we checked at random 15% of the data provided in the RevMan file with the data published of the relevant studies. No errors were found at this stage. The RevMan tables were thus updated with new data. Additional data allowed for new subgroup analyses. Potential errors in the weighted mean difference (WMD) in systolic blood pressure (SBP) between the older trials compared with the newer trials were identified. Methods: We reextracted data on SBP from the original 38 studies, and this was double checked with the original data entered in RevMan (taking three hours). Twenty errors (8.8%) out of a potential 228 pieces of data were found in the SBP table. The most common error was transposition of standard deviation change. Corrections to the tables and re-analysis gave us a WMD for SBP of -2.71 [-3.49 to -1.93] compared to the original estimate of -6.73 [-9.72 to -3.73]. To ensure that all data was correct, we reextracted data on the remaining nine outcomes (taking 13 hours). Further errors in the data tables were found in deaths, diastolic blood pressure and cholesterol, however, the WMD differences between the original and the revised analysis were not great. Blind double entry of the data took us < 20 hours, and errors included five studies being placed in the wrong category and a number of minor inputting errors. Results: Overall, it took us 36 hours to double check 7200 separate pieces of data from the original papers and perform blind double entry from the 55 studies included in both the original review and the update. Conclusions: We are now confident the data is accurate. We urge authors undertaking similar updates of systematic reviews to be vigilant and consider checking previous data. Our results indicate an essential need for guidance on updating systematic reviews.