Are all Cochrane reviews born equal? Methodological quality across different Review Groups

Article type
Authors
Pantoja T1, Moreno G1, Soto M1
1Family Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
Abstract
Background: High quality systematic reviews are increasingly recognized as providing the best evidence to inform healthcare decisions at different levels. The quality of a review and then the confidence that a decision maker puts on its findings depends on the extension to which its design will generate unbiased results. Published reviews vary considerably on their quality, with Cochrane reviews rating consistently better than non-Cochrane ones. However, little is known about the quality of reviews across different Review Groups in the Collaboration. Considering that the review production process in the Collaboration is highly decentralized some variation is expected, but its magnitude is not known. Objective: To compare the methodological quality of systematic reviews produced by different Cochrane Review Groups. Methods: We selected a random sample of reviews published in the February 2010 issue of the Cochrane Library. Each selected review was independently assessed by two evaluators using the AMSTAR tool. The percentage of reviews from each Review Group meeting each of the eleven AMSTAR items was calculated. Likewise the proportion of reviews from each Review Group meeting a high-quality score was calculated. Data were analyzed using non-parametric approaches. Results: In our initial assessment of a sub-sample of reviews there were some differences in the quality of them. However, we have not completed the assessment of the full sample in order to attribute that difference to the Review Groups where they were produced. Conclusions: It might be differences in the methodological quality of reviews produced by different Review Groups. If this is verified in a the analysis of the whole sample of reviews more work should be done within the Collaboration to assure appropriate quality standards of its main product.