Quality of Cochrane systematic review abstracts; readability and comparison with handbook guidelines

Tags: Oral
Magarey A, Silagy C, Middleton P, Bastian H, Middleton P

Introduction/Objective: To assess the quality of abstracts prepared for Cochrane reviews and the degree to which they meet the guidelines provided in the Cochrane handbook.

Methods: One abstract in forty of the new Cochrane reviews in Cochrane Library 1,1998 was compared against the handbook guidelines, supplemented with further detail from articles by Mulrow et al on the rationale for structured abstracts. Readability (using the Flesch reading ease index) was measured. While this measure of readability has limitations when applied to scientific writing, it gives a broad reading ease indication. Consistency of information within and between abstracts was also examined.

Results: Seventy-five percent of abstracts (n= 30/40) had an average word length longer than 250 words, the recommended length. Other significant departures from the guidelines were found in the following sections: Population was omitted in 29 abstracts; Search strategy - time restrictions omitted in 32 abstracts; Data collection and analysis - guidelines for assessing data quality omitted in 31 abstracts. The average readability score of the 40 abstracts was 28.8. This represents a reading level accessible to less than 30% of the adult English-speaking population and about half that of standard writing which has a score of 60-70. Inconsistency between reviews was most apparent in the Data collection and analysis section, with some abstracts outlining what data was extracted and others outlining how data was abstracted. The Results section often contains additional information apart from the qualitative and quantitative results. There is also inconsistency in how odds ratios and confidence intervals are presented.

Discussion: There is considerable variation in length, readability and consistency of Cochrane Library abstracts. While most abstracts include the core information suggested by the Cochrane handbook, this could be achieved in a more concise, consistent and accessible fashion, by reviewers and editors adhering to current handbook guidelines and by checking the readability of their abstracts. It would also be useful to expand on the handbook guidelines for preparing abstracts and some suggestions for this are made.