Background: High quality systematic reviews are increasingly recognized as providing the best evidence to inform healthcare decisions at different levels. The quality of a review— and then the confidence that decision-makers put on its findings— depends on the extension to which its design will generate unbiased results. Published reviews vary considerably in their quality, with Cochrane reviews rating consistently better than non-Cochrane reviews. However, little is known about the quality of reviews across different Review Groups in the Collaboration. Considering that the review production process in the Collaboration is highly decentralized some variation is expected, but its magnitude is not known.

Objective: To compare the methodological quality of systematic reviews produced by different Cochrane Review Groups.

Methods: All new reviews published in issues 3 and 4, 2011 of The Cochrane Library were selected for appraisal. Each selected review was independently assessed by two evaluators using the AMSTAR tool. The percentage of reviews from each Review Group that met each of the 11 AMSTAR items and the proportion that met a 'high-quality’ score were calculated. Data were analyzed using non-parametric approaches.

Results: Fifty reviews were selected for appraisal (31 from issue 3;19 from issue 4). In our initial assessment of a subsamplesome differences in 'quality’ were found. However, we have not completed the assessment of the full sample in order to attribute those differences to the Review Groups where they were produced.

Conclusions: It might be differences in the 'methodological quality’ of reviews produced by different Review Groups. If this is verified in the analysis of the whole sample of reviews more work should be done within the Collaboration to assure appropriate quality standards of its main product through changes in the editorial processes of review groups.