How empty are empty reviews? A report on empty reviews in the CDSR and an invitation to participate in the development of reporting guidelines

Article type
Authors
Yaffe J1, Montgomery P2, Shepard L2, Hopewell S3
1College of Social Work, University of Utah, Salt Lake City, Utah, United States
2Centre for Evidence-Based Intervention, Oxford University, Oxford, UK
3UK Cochrane Centre, Oxford, UK
Abstract
Background: Users rely on The Cochrane Database of Systematic Reviews (CDSR)1,2,3,4,5 for the best available evidence in making health and social care decisions. However, some evidence suggests that ‘‘empty reviews’’ in the CSDR provide limited implications for practice or make recommendations without consideration of bias in excluded studies. Objectives: To report our findings on the reporting of empty reviews, to seek input for the development of recommended guidelines for reporting empty reviews to the Cochrane Methods Group, and to invite participation in the guideline development process. Methods: Following the suggestions of Moher et al.6 for the development of reporting guidelines, this project has identified the need for reporting guidelines for reviews without included studies by reviewing the literature, identifying all empty reviews within the CDSR, describing these reviews, examining how these reviews report implications for practice and consulting with other systematic review groups to find existing guidelines for reporting empty reviews. This information is presented for discussion in this forum to further prepare for focus groups with Cochrane/Campbell users and a formal guideline consensus meeting with invited Cochrane/Campbell contributors. Results: At present the reporting of empty reviews in the CDSR is inconsistent and the Cochrane Handbook for Systematic Reviews of Interventions does not provide reporting guidelines. Through Issue 3, 2010, there are 374 empty reviews reported in the CDSR, representing 9% of total published reviews and 0% to 40% of reviews by Cochrane Review Group (Figure 1). The publication of empty reviews appears to be increasing over time (Table 1). Conclusions: Guidelines for the reporting of empty reviews are needed. We invite the open discussion of Cochrane and Campbell contributors regarding recommendations towards the development of guidelines. We hope to encourage participation by interested colleagues in a future, more formal consensus meeting.

References : 1. LANG, A., EDWARDS, N. and FLEISZER, A., 2007. Empty systematic reviews: hidden perils and lessons learned. Journal of clinical epidemiology, 60(6), 595–597.
2. GREEN, S., HIGGINS, J.P.T., SCHU¨ NEMANN, H.J. and BECKER, L., 2007. Response to paper by Lang A, Edwards N, and Fleiszer A. Journal of clinical epidemiology, 60(6), 598–599.
3. PAGLIARO, L., BRUZZI, P. and BOBBIO, M., 2010. Why are Cochrane hepato-biliary reviews undervalued by physicians as an aid for clinical decision-making? Digestive and liver disease, 42(1), 1–5.
4. SCHLOSSER, R.W. and SIGAFOOS, J., 2009. ‘Empty’ reviews and evidence-based practice. Evidence-based communication assessment and intervention, 3(1), 1–3.
5. GOTZSCHE, P. Far too many excluded studies listed in Cochrane reviews [abstract]. XIV Cochrane Colloquium; 2006 October 23–26; Dublin, Ireland.
6. MOHER, D., SCHULZ, K.F., SIMERA, I. and ALTMAN, D.G., 2010. Guidance for developers of health research reporting guidelines. PLoS medicine, 7(2).