Quality assurance of Cochrane systematic review abstracts; a comparison with abstracts published in evidence-based medicine

Tags: Oral
Silagy C, Middleton P, Magarey A, Bastian H

Introduction/Objective: To compare the quality of abstracts prepared for Cochrane reviews by the authors of the reviews compared with those produced for the same review by a professional writing team associated with the international journal Evidence-Based Medicine (EBM).

Methods: All Cochrane reviews selected for publication and commentary in EBM (since its inception in 1995 up to and including vol 3 (2) March/April 1998) were included in the study. For each of the Cochrane reviews, we obtained the abstract from the version of the Cochrane Library (CL) which had been used to prepare the abstract and commentary for EBM. The number of words in each section of the two versions of the abstracts (CL or EBM) was compared, as were the readability scores (using the Flesch Reading Ease index). Two of the authors then independently read both versions of the abstracts and recorded whether there had been a change in the quantity and meaning of the information provided in the EBM version compared with the original CL version.

Results: A total of 15 Cochrane reviews met the selection criteria. The overall number of words were greater for the CL version of the abstracts, compared with those appearing in EBM (average no. of words per abstract: 378 CL, 330 EBM). The readability scores were also lower for the CL abstracts (average readability scores: 33.6 CL. 35.9 EBM). In 60% (n=9/15) of the reviews the amount of information provided in the EBM version had increased relative to the original CL version, whereas 40% (n=6/15) abstracts showed a decrease. Sections showing the most changes in content were the title, selection criteria, results and conclusions. The EBM results section was frequently, extensively rewritten and often supplemented with tables of summary measures, other than odds ratios. The EBM abstracts were more consistent in style and in the types of information contained in each section of the abstract. The meaning and intent of the different sections remained largely unchanged between the two versions, except for titles and data collection and analysis.

Discussion: The use of a professional editorial writing team improves the consistency and readability and hence the accessibility of abstracts. Although the content of the various sections of the abstracts was frequently altered, this usually clarified the abstract without changing the meaning. Though, the readability and word length improvements were modest, further gains may be possible for the abstracts published in CL. Given the importance many readers attach to the abstract, the information from this study suggests ways of improving the quality assurance of abstracts prepared, in association with reviews appearing in the CL.