One in three systematic reviews published in high-impact medical journals assessed the certainty of evidence, though frequency increased over time

Poster
2024 Prague [Global Evidence Summit]
Siedler M1, Tangri N2, AlShenaiber L3, Pasumarthi T4, Ali F5, Gaby V6, Harris K7, Falck-Ytter Y8, Mustafa R6, Sultan S2, Dahm P2, Murad M, Morgan R
1Evidence Foundation, Cleveland Heights, Ohio, United States; Department of Kinesiology and Sport Management, Texas Tech University, Lubbock, Texas, United States
2Evidence Foundation, Cleveland Heights, Ohio, United States
3Evidence Foundation, Cleveland Heights, Ohio, United States; College of Medicine and Health Sciences, Khalifa University, Abu Dhabi, United Arab Emirates
4Evidence Foundation, Cleveland Heights, Ohio, United States; School of Interdisciplinary Science, McMaster University, Hamilton, Ontario, Canada
5Department of Gastroenterology, Hepatology and Nutrition, The University of Texas Health Science Center, Houston, Texas, United States
6Evidence Foundation, Cleveland Heights, Ohio, United States; Department of Health Research Methods, Evidence, and Impact (HEI), McMaster University, Hamilton, Ontario, Canada
7Department of Kinesiology and Sport Management, Texas Tech University, Lubbock, Texas, United States
8Evidence Foundation, Cleveland Heights, Ohio, United States; School of Medicine, Case Western Reserve University, Cleveland, Ohio, United States
"

BACKGROUND:

Certainty of evidence (COE) assessment is a key part of a rigorous and transparent systematic review. In 2020, COE assessment became a standalone item within the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidance. It is unknown how frequently systematic reviews published in high-impact medical journals assess COE and whether this proportion has changed over time.

OBJECTIVES:

To examine the prevalence of COE assessment in systematic reviews published in high-impact medicine journals over the past 11 years and to investigate whether the frequency of COE assessment in these journals has changed over time.

METHODS:

A PubMed search with hand-searching of relevant journal websites was used to identify systematic reviews published between 24 January 2013 and 23 January 2024. Two reviewers independently screened any systematic review published in one of the top-ten highest-impact journals within the 2021 Journal Citation Report for the General and Internal Medicine category to determine whether COE was assessed using any method. COE assessment was defined as the combination of two or more conceptual domains (e.g., risk of bias, consistency, directness, etc.) informing an overall rating or statement about the confidence or trustworthiness of the evidence. Binomial regression examined year of publication to determine whether COE assessment changed over time. No patients or healthcare consumers were involved in this project.

RESULTS:

Of 1,030 eligible systematic reviews, 357 (34.7%) assessed COE. Seven of the ten journals had published at least one article assessing COE. Across these journals, the proportion of eligible systematic reviews assessing COE ranged from 6.8% to 48.8% (Table 1). COE assessment generally increased over time, ranging from 12.5% of eligible reviews in 2014 to 54.2% in 2022 (Table 2). Year of publication was a statistically significant predictor of COE assessment (coefficient: 0.16; p<0.001; Figure 1).

CONCLUSIONS:

Only one in three systematic reviews published in the highest-impact medical journals over the past 11 years assessed COE, which is key for informing evidence-based patient care. The likelihood of COE assessment in these journals appears to be slowly increasing year-to-year, perhaps due to increased education, manuscript submission requirements, and dissemination of reporting standards."