A readability assessment of Plain Language Summaries and abstracts of Cochrane Reviews before and after guidance of standards

Poster
2024 Prague [Global Evidence Summit]
Briceño F1, Meza N2, Metzendorf M3, Garnham R4, Casino G5, Cervera A5, Franco J3, Escobar-Liquitay C6, Bracchiglione J7, Urrútia G8, Madrid E2
1School of Medicine Universidad de Valparaíso, Viña del Mar, Chile
2Interdisciplinary Centre for Health Studies (CIESAL), Cochrane Chile Associate Centre Universidad de Valparaíso Chile, Valparaíso, Chile; Iberoamerican Cochrane Center - Institut de Recerca Sant Pau (IR SANT PAU) , Barcelona, Spain
3Institute of General Practice, Medical Faculty of the Heinrich Heine University , Düsseldorf, Germany
4Interdisciplinary Centre for Health Studies (CIESAL), Cochrane Chile Associate Centre Universidad de Valparaíso Chile, Valparaíso, Chile
5Iberoamerican Cochrane Center - Institut de Recerca Sant Pau (IR SANT PAU) , Barcelona, Spain
6Research Department, Associate Cochrane Centre, Instituto Universitario Escuela de Medicina del Hospital Italiano , Buenos Aires, Argentina
7Interdisciplinary Centre for Health Studies (CIESAL), Cochrane Chile Associate Centre Universidad de Valparaíso Chile, Valparaíso, Chile; Iberoamerican Cochrane Center - Institut de Recerca Sant Pau (IR SANT PAU) - CIBERESP, Barcelona , Barcelona, Spain
8Iberoamerican Cochrane Center - Institut de Recerca Sant Pau (IR SANT PAU) - CIBERESP, Barcelona , Barcelona, Spain
"

Background:

Plain Language Summaries (PLS) of Cochrane Reviews (CR) must be written in clear and straightforward language to serve the aim of knowledge translation. Unlike abstracts—aimed for dissemination among professionals with science literacy—PLS should accurately summarise results in a succinct and readable style to target broad audiences. Cochrane has standards for elaborating the message of PLS, and both editors and authors should ensure these aspects. The Cochrane Handbook advises that PLS are written for a reading age of around 11. Cochrane developed standards in a participative manner to guarantee the appropriateness of PLS and released guidance in 2022. No assessments exist on its accomplishment.

Objectives:

To analyse the readability of the PLS and abstracts of CR of interventions before and after the newest Cochrane guidance.

Methods:

Cross-sectional study to assess all PLS and abstracts of CR of interventions published in 2019 (before guidance) and 2023 (after guidance) and compare their readability. We excluded protocols, withdrawn and non-intervention reviews. We extracted the authors´country of affiliation, and text of the PLS and the abstract (including titles). We used the ‘Readability Formula’ suggested by MECIR, to obtain the Simple Measure of Gobbledygook Index (SMOG) and other indices, which estimate the years of education needed to understand a text.

Results:

We analysed 546 CR published in 2019 and 415 published in 2023. Before guidance implementation, PLS were significantly less readable than abstracts (Abstracts’ SMOG=11.32±1.55; PLS SMOG=11.99±1.73; p<0.001)). In this period, 326 (59.7%) of the CR had corresponding authors from native English-speaking countries (MNESC). We found no differences between the readability of PLS or abstracts, according to whether the authors’ affiliations were based in MNESC or not. Our preliminary analysis of 2023 PLS and abstracts shows similar results.

Conclusions:

The abstracts and PLS of CR published during 2019 have a readability score for lay people equivalent to 11th grade (i.e. 16 to 17 years old), which may be appropriate for abstracts but inappropriate for PLS, that aim to translate knowledge to broad audiences. Improving the readability of PLSs is still a pending challenge. We will present detailed comparisons between 2019 and 2023 at GES 2024."