Background:

by publishing research protocols and defining hypothesis prior to conducting a study and analyzing the results, it is possible to minimize the risk of bias associated with a given study.

Objectives:

we compared published non-Cochrane systematic reviews with their registered protocols in the International Prospective Register of Systematic Reviews (PROSPERO) to determine which changes were made and how likely these changes were to have resulted in the introduction of bias.

Methods:

in this retrospective comparative study, we electronically searched for protocols and their corresponding reviews in the PROSPERO database that were 'completed and published' from January to September, 2018. Two review authors independently identified and classified changes between the protocols and reviews as none, partial, or complete.

Results:

we identified 43 protocols with their corresponding published systematic reviews, all of which (100%) exhibited a change compared with the published protocol. Only eight of 43 (19%) exhibited only a partial change, the remaining 81% of reviews (n = 35) consisted of at least one section that had undergone complete change. The changes made in each review are shown in Figure 1. The greatest variation between reviews and their protocols was in the search strategy, with changes in 41 reviews (95%). A summary of the section changes is shown in Table 1. Some changes have a potential risk of introducing bias, such as expanding/modifying the review question, decreasing search database/narrowing the time range of retrieval, changing inclusion criteria/exclusion criteria, adding/deleting outcomes or outcome measures.

Conclusions:

non-Cochrane systematic reviews that are registered in PROSPERO are not fully consistent with previously published protocols, and some changes may introduce bias. The registration process of PROSPERO does not conduct peer reviews or quality assessments, which in turn affects the transparency of reviews. Thus, PROSPERO needs to rigorously examine submitted protocols and introduce peer review to improve the quality of registration. Furthermore, the publication of appendices, which documented the changes and reasons between the protocols and the full text, could be gradually promoted and encouraged to strengthen transparency of the systematic review further.

Patient or healthcare consumer involvement: systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of healthcare interventions accurately and reliably. Given that systematic reviews are generally promoted as providing the best evidence for medical decision makers to make clinical decisions for patients, it is important to promote the reviews as being as transparent and bias-free as possible.