Four years’ feedback submitted through The Cochrane Library: what happened to it?

Article type
Authors
Carlisle J
Abstract
Background: Feedback can improve systematic reviews by identifying errors in methods, data extraction, analyses, unfounded assumptions leading to flawed conclusions and other areas. Readers can comment on Cochrane protocols and reviews electronically through the Library website. Managing Editors and feedback editors are forwarded these comments by Wiley. Substantive comments on methodology, results or conclusions should be subsequently published within the ‘Feedback’ section of the review or protocol along with review authors’ responses. Objectives: To compare comments submitted through Wiley’s Cochrane Library website with those published in the ‘Feedback’ section of reviews. Methods: I accessed Wiley’s archive of comments submitted through The Cochrane Library (www.cochranefeedback.com/ cf/cma). Wiley staff forward comments they categorize ‘Approved’. I could not be sure other comments were forwarded to CRGs (categorized ‘Draft’, ‘Not Approved’ and ‘Deleted’). I read all the comments. I then read the reviews and counted which had published the feedback and which had not. I calculated how much time had passed since comments were submitted. Results: 511 comments were submitted over 4 years. Of these, 397 were categorized ‘Approved’, 342 of which I thought were substantive: 104 (30%) were published in the ‘Feedback’ section of reviews and two were entered into another section of the review. Authors had responded to 89 of these 104 comments. The median time that had elapsed since the 236 unpublished comments had been forwarded to CRGs was 15 months (mean 16.3 months). Five CRGs had received no comments on their reviews. The Pregnancy and Childbirth Group received the most (n=49). Eight CRGs had published all the comments they had received, of which the Multiple Sclerosis Group had received the most (n=3). Conclusions: Cochrane Review Groups receive comments on their reviews through a variety of routes, one of which is the Library website. This is the only route that allows The Cochrane Collaboration to assess the quality of the feedback process, facilitating improvements in reviews that are published and those that are not yet published. The Feedback Management Advisory Group (FMAG) has previously considered ways of encouraging submission of comments through the Library website. Until CRGs improve their management of comments the FMAG should not promote electronic submission of feedback.