Methods to reduce reporting bias in Cochrane reviews

Article type
Authors
Mann M1, Rader T2, Stansfield C3, Cooper C4, Sampson M5
1Cardiff University, UK
2Cochrane Musculoskeletal Group, Canada
3EPPI-Centre, Institute for Education, UK
4Peninsula Technology Assessment Group (PenTAG), UK
5Children’s Hospital of Eastern Ontario Research Institute, Canada
Abstract
Background: The value of a systematic review depends on the clarity of reporting. Rigorous reporting of the search process can help review authors avoid potential bias in their information retrieval. However little has been published on the reasons for potential disparities in reporting and the best practices for documentation.

Objectives: To explore current practice and perceived challenges in search documentation and to assess potential for bias in reporting the search.

Methods: An online survey was conducted among systematic review authors, Cochrane Trials Search Co-ordinators, librarians and other information specialists during February/March 2011. A total of two hundred and sixty-three responded, though not all answered the questions.

Results: The survey revealed a disparity in the amount of information that is documented and reported. For example, approximately 15% of respondents do not report the search platform or the timeframe the literature is searched. While 99% of respondents (n = 196) said they record the database name and one search strategy, only 56% (n = 111) indicate the coverage of the database by recording the date, number or issue of the database. Tools and methods to record these elements varied widely from reference management software, to excel spreadsheets and manual methods. Similarly; the varied responses to the open ended question, ‘What aspects do you find most difficult?’ (n = 141) will be presented.

Conclusions: Clarity and the ability to reproduce the search should drive the documentation process and ultimately help the reader assess the comprehensiveness of the search which forms the basis of the systematic review.