Are Cochrane Review Protocols available and post-protocol changes to methods documented? A study in HIV/AIDS and ophthalmology

Article type
Authors
Saldanha I1, Li T1, Heyward J2, Dickersin K1
1Cochrane Eyes and Vision Group/US Cochrane Center/Johns Hopkins Bloomberg School of Public Health, USA
2Cochrane Eyes and Vision Group/US Cochrane Center, USA
Abstract
Background: There is widespread acknowledgement that clinical trial protocols & amendments should be made public. It is unclear if comparable access to systematic review (SR) protocols & amendments is available. Objectives: To evaluate the availability of Cochrane SR Protocols, extent to which they are updated, and reporting of post-protocol changes to methods in completed SRs. Methods: We examined all Cochrane SRs (as of 30 June 2013) on HIV/AIDS & four common eye conditions. We estimated the proportion of SRs for which the protocol was available (using the 'Other Versions' tab in The Cochrane Library), number of updates to each protocol, and duration between publication of protocol & most recent version of the completed SR. For each completed version, we extracted from the 'History', 'What’s new', and 'Differences between protocol & completed review' ('Differences') sections information about post-protocol changes to methods. Results: We included 197 SRs (Table 1). Protocols were available for 160/197 (81.2%), and updated protocols were available for 13/197 (6.6%). Twelve of the 13 updated protocols reported authorship changes and one reported an inclusion criterion change. Among all 136 completed SRs (Fig 1), the first completed version of 89 SRs (65.4%) did not discuss post-protocol changes to methods, while 47 SRs (34.6%) discussed changes (29 reported changes & 18 explicitly reported that there were no changes) (Table 2). In all five instances where methods were changed & another update of the completed SR was available, the update listed all prior changes. Conclusions: Protocols for almost 1/5 Cochrane SRs were unavailable in The Cochrane Library. Approximately 2/3 completed SRs did not report whether post-protocol changes to methods were made. The Handbook & MECIR do not state what authors should report if there are no changes. We can’t tell whether any post-protocol changes were made to methods for most of the Cochrane SRs. Even when authors note methods changes in the 'Differences' section, it is unclear whether the changes are specified before the update is made. A solution is needed that satisfies the needs of SR authors, users, and researchers.