Background: The protocol is considered an important element when a study is conducted. Firstly, it makes a study replicable by independent researchers, secondly it provides the researchers with a guide for the conduct of the study; finally it protects from biases due to pre-specifying hypotheses of the research. Evidence suggests that there are discrepancies between protocols and final studies in terms of outcomes in primary literature1
Objectives: To evaluate whether discrepancies exist between outcomes and systematic review (SR) protocols in Cochrane SRs.
Methods: All new SRs published in the 2005 CL Issues 2, 3, 4 and in 2006 CL Issue 1 were identified. A random sample of SRs was selected. For each SR the most recent version of the protocol was retrieved. Using an ad hoc extraction form, previously developed and validated, data from protocols and SRs were extracted by a group of trained reviewers. To maintain the assessment blind, reviewers were assigned to unpaired protocols and SRs. In protocols, information regarding the definition of outcomes of effectiveness and adverse events (AE) and their typologies (primary/secondary/not defined) was collected. The same information was collected in the SRs. Agreement between protocols and SRs in terms of outcomes were eventually assessed and the percentage of disagreement estimated. Disagreement was operationally defined as discrepancies between SRs and their protocols which were not motivated by the authors in their SRs. When disagreement occurred in a SR it was classified into disagreement due to: a) inclusion of at least a new outcome which was not defined in the protocol; b) exclusion of at least one outcome which was defined in the protocol; c) change in types of outcome. This type of disagreement was dichotomised as upgrading and downgrading change. A change was classified as "upgrading" when outcomes defined as secondary in the protocol became primary in the SR or when not defined in the protocol became secondary or primary in the SR. A change was classified as "downgrading" when outcomes stated as primary or secondary in the protocol became secondary or not defined in the SR, respectively.
Results: This analysis is based on a sample of 60 SRs. Outcomes of AE were reported in 39 SRs (65%). The disagreement with protocols was observed in 28 SRs (47%) for effectiveness outcomes and in 9 SRs (23%) for AE outcomes. Eleven SRs (18%) included at least a new outcome, 17% excluded at least 1 outcome. In those 18 SRs (30%) in which a change in type of outcome was observed 6 (10%) upgraded at least 1 outcome, 9 (15%) downgraded at least one outcome, 3 (5%) did both. Similar results were observed for AE outcomes. The median number of outcomes per protocol was 9 and it was 11 in SRs (range 2 to 45). We are currently analysing the association between type of change and statistical significance. The results will be presented considering a larger sample size.
Conclusions: Our preliminary results show that discrepancies between protocols and SRs are not uncommon. Most discrepancies are due to change in the typologies of outcomes: many outcomes that were primary in the protocols became secondary, or undefined, in the SRs. Possible reasons for this phenomenon will be presented and discussed at the Colloquium.

References: 1) Chan A, Hrobjartsson A, Haahr M, Gotzsche P, Altman D: Empirical evidence for selective reporting of outcome in randomised trials. JAMA, 2004;291:2457-65.