Article type
Abstract
"BACKGROUND: Certainty of evidence (COE) assessment is a key part of a rigorous and transparent systematic review. In 2020, COE assessment became a standalone item within the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidance. It is unknown how frequently systematic reviews published in high-impact medical journals assess COE and whether this proportion has changed over time.
OBJECTIVES: To examine the prevalence of COE assessment in systematic reviews published in high-impact medicine journals over the past 11 years and to investigate whether the frequency of COE assessment in these journals has changed over time.
METHODS: A PubMed search with hand-searching of relevant journal websites was used to identify systematic reviews published between 24 January 2013 and 23 January 2024. Two reviewers independently screened any systematic review published in one of the top-ten highest-impact journals within the 2021 Journal Citation Report for the General and Internal Medicine category to determine whether COE was assessed using any method. COE assessment was defined as the combination of two or more conceptual domains (e.g., risk of bias, consistency, directness, etc.) informing an overall rating or statement about the confidence or trustworthiness of the evidence. Binomial regression examined year of publication to determine whether COE assessment changed over time. No patients or healthcare consumers were involved in this project.
RESULTS: Of 1,030 eligible systematic reviews, 357 (34.7%) assessed COE. Seven of the ten journals had published at least one article assessing COE. Across these journals, the proportion of eligible systematic reviews assessing COE ranged from 6.8% to 48.8% (Table 1). COE assessment generally increased over time, ranging from 12.5% of eligible reviews in 2014 to 54.2% in 2022 (Table 2). Year of publication was a statistically significant predictor of COE assessment (coefficient: 0.16; p<0.001; Figure 1).
CONCLUSIONS: Only one in three systematic reviews published in the highest-impact medical journals over the past 11 years assessed COE, which is key for informing evidence-based patient care. The likelihood of COE assessment in these journals appears to be slowly increasing year-to-year, perhaps due to increased education, manuscript submission requirements, and dissemination of reporting standards."
OBJECTIVES: To examine the prevalence of COE assessment in systematic reviews published in high-impact medicine journals over the past 11 years and to investigate whether the frequency of COE assessment in these journals has changed over time.
METHODS: A PubMed search with hand-searching of relevant journal websites was used to identify systematic reviews published between 24 January 2013 and 23 January 2024. Two reviewers independently screened any systematic review published in one of the top-ten highest-impact journals within the 2021 Journal Citation Report for the General and Internal Medicine category to determine whether COE was assessed using any method. COE assessment was defined as the combination of two or more conceptual domains (e.g., risk of bias, consistency, directness, etc.) informing an overall rating or statement about the confidence or trustworthiness of the evidence. Binomial regression examined year of publication to determine whether COE assessment changed over time. No patients or healthcare consumers were involved in this project.
RESULTS: Of 1,030 eligible systematic reviews, 357 (34.7%) assessed COE. Seven of the ten journals had published at least one article assessing COE. Across these journals, the proportion of eligible systematic reviews assessing COE ranged from 6.8% to 48.8% (Table 1). COE assessment generally increased over time, ranging from 12.5% of eligible reviews in 2014 to 54.2% in 2022 (Table 2). Year of publication was a statistically significant predictor of COE assessment (coefficient: 0.16; p<0.001; Figure 1).
CONCLUSIONS: Only one in three systematic reviews published in the highest-impact medical journals over the past 11 years assessed COE, which is key for informing evidence-based patient care. The likelihood of COE assessment in these journals appears to be slowly increasing year-to-year, perhaps due to increased education, manuscript submission requirements, and dissemination of reporting standards."