Article type
Year
Abstract
Background: Imprecision understood as the risk of random error is one of the five dimensions assessed when rating down the certainty of the evidence for a determined outcome. Along with risk of bias, imprecision is the most common domain associated with Grading of Recommendations Assessment, Development and Evaluation (GRADE) system for estimating overall certainty of evidence (CoE). However, the rationale for assessing imprecision seems to be inconsistently reported among authors of Cochrane systematic reviews (CSRs).
Objectives: We aimed to evaluate the report of imprecision and the reasons for downgrading certainty of evidence in CSRs of intervention published during 2019.
Methods: This cross-sectional study aimed to analyse the rationale behind imprecision assessments by authors of CSRs. We included all CSRs of intervention published during 2019 with at least one Summary of Findings (SoF) table. We excluded non-intervention reviews (such as qualitative or diagnostic test accuracy reviews), empty reviews, overviews and methodological reviews. We extracted information from the first reported SoF table, including footnotes and comments section, for each outcome, and we summarised findings descriptively using absolute and relative frequencies.
Results: We included 499 CSRs. In 438 (87.7%) reviews the authors downgraded the CoE of at least one outcome due to imprecision (Figure 1). Among these, 355 (81.0%) CSRs explicitly stated a downgraded CoE due to imprecision, whereas 83 CSRs (18.9%) provided a rationale for this, but did not explicitly mention the domain imprecision. The most common reasons for downgrading the CoE due to imprecision were “few events/patients or small sample size” (291 CSRs, 66.4%), “wide confidence intervals” (181 CSRs, 41.3%) and “cross the line of no effect” (150 CSRs, 34.2%). Only 48 CSRs (10.9%) used the concept of optimal information size. In 32 (7.31%) CSRs that explicitly downgraded due to imprecision, the authors did not provide a sufficiently clear rationale for their decision.
Conclusions: Imprecision is a common reason for downgrading CoE among CSRs. Authors usually justify this assessment arguing the low number of events or patients, and the width of confidence intervals. However, an important proportion of CSRs do not justify the reasons for this downgrading and some reviews provided reasons that might not be adequate.
Patient or healthcare consumer involvement: No patients were involved in the development of this research.
Objectives: We aimed to evaluate the report of imprecision and the reasons for downgrading certainty of evidence in CSRs of intervention published during 2019.
Methods: This cross-sectional study aimed to analyse the rationale behind imprecision assessments by authors of CSRs. We included all CSRs of intervention published during 2019 with at least one Summary of Findings (SoF) table. We excluded non-intervention reviews (such as qualitative or diagnostic test accuracy reviews), empty reviews, overviews and methodological reviews. We extracted information from the first reported SoF table, including footnotes and comments section, for each outcome, and we summarised findings descriptively using absolute and relative frequencies.
Results: We included 499 CSRs. In 438 (87.7%) reviews the authors downgraded the CoE of at least one outcome due to imprecision (Figure 1). Among these, 355 (81.0%) CSRs explicitly stated a downgraded CoE due to imprecision, whereas 83 CSRs (18.9%) provided a rationale for this, but did not explicitly mention the domain imprecision. The most common reasons for downgrading the CoE due to imprecision were “few events/patients or small sample size” (291 CSRs, 66.4%), “wide confidence intervals” (181 CSRs, 41.3%) and “cross the line of no effect” (150 CSRs, 34.2%). Only 48 CSRs (10.9%) used the concept of optimal information size. In 32 (7.31%) CSRs that explicitly downgraded due to imprecision, the authors did not provide a sufficiently clear rationale for their decision.
Conclusions: Imprecision is a common reason for downgrading CoE among CSRs. Authors usually justify this assessment arguing the low number of events or patients, and the width of confidence intervals. However, an important proportion of CSRs do not justify the reasons for this downgrading and some reviews provided reasons that might not be adequate.
Patient or healthcare consumer involvement: No patients were involved in the development of this research.