Background: Medical students, nursing students, pharmacy students and other health professional students have heard of the Cochrane Collaboration, but seldom access reviews to answer clinical questions. We have tried to encourage our students to do so, but encountered significant problems.
Objectives: Encourage health professional students to utilize the scientific knowledge available in Cochrane systematic reviews and to understand both its strenghts and limitations.
Methods: Since 2005 we have routinely taught elective medical students and other health professional or science students and post-graduate trainees how to read and evaluate Cochrane systematic reviews. At the same time, we have introduced them to critical appraisal of the primary RCT reports, and encouraged students to assess whether Cochrane systematic review authors have followed standard Cochrane methodology so as to reach believable conclusions. Over this interval, understanding of publication bias and other biases has expanded, so we have placed increasing evidence on assessing the quality of systematic reviews and of the authors’ critical appraisal of the primary evidence.
Results: Our experience has shown that Cochrane systematic reviews vary in quality from very useful and apparently reliable to virtually irrelevant to the apparent clinical issue or clearly unreliable. We will show examples where students could recognize flaws that apparently evaded the authors, peer reviewers, and Cochrane editorial groups, as well as examples which encouraged students to access Cochrane Reviews again to explore evidence as opposed to ‘expert opinion’, ‘consensus’, or guidelines..
Conclusions: Cochrane systematic reviews vary substantially in quality. While a high quality review may encourage health professionals to utilize this unique source of scientific information, low quality reviews can discourage even a Cochrane enthusiast. Authors and editorial groups need to listen the experience of systematic review users in order to ensure that the Cochrane Collaboration meets its idealistic goals.