Article type
Year
Abstract
Background: Assessing the quality of the body of evidence (QoE) using GRADE criteria and justifying and documenting these assessments are highly desirable for new Cochrane Reviews as part of Summary of Findings (SoF) tables (MECIR standards). Although—in theory— possible, examples for upgrading a body of evidence from RCTs are elusive. Our experience in working with authors of systematic reviews and guideline developers is that interpretation of this guidance is often challenging.
Objectives: To summarize if, and how Cochrane authors use upgrading the quality of RCT evidence in Cochrane Reviews.
Methods: We reviewed all SoF tables published in the Cochrane Library Issue 3, 2012 (n = 502) and selected those tables in which the quality of RCT evidence was upgraded. These were studied in detail to seewhether upgrading was appropriate (using the criteria of the GRADE JCE paper on upgrading).
Results: In three SoF tables upgrading of downgraded RCTs was used explicitly, in six SoF tables implicitly. In eight tables the QoE was upgraded because of a large magnitude of effect; in one because of a dose-response association. In all nine tables risk of bias was the downgrading factor (unclear allocation concealment, incomplete outcome assessment, early stopping, no ITT analysis, outcome reporting bias), in one table in combination with imprecision and in two tables in combination with heterogeneity. Studying these SoF tables revealed that in all cases upgrading was not appropriate or at least questionable. In some SoF tables downgrading for imprecision was overlooked.
Conclusions: Upgrading of RCTs affected by study limitations was considered appropriate by Cochrane authors in 2% of the reviewed SoF tables, mostly because of a large magnitude of effect. According to the current GRADE guidance on upgrading, the upgrading was at least questionable. More detailed guidance and worked out examples are needed. These will be presented at the Colloquium.
Objectives: To summarize if, and how Cochrane authors use upgrading the quality of RCT evidence in Cochrane Reviews.
Methods: We reviewed all SoF tables published in the Cochrane Library Issue 3, 2012 (n = 502) and selected those tables in which the quality of RCT evidence was upgraded. These were studied in detail to seewhether upgrading was appropriate (using the criteria of the GRADE JCE paper on upgrading).
Results: In three SoF tables upgrading of downgraded RCTs was used explicitly, in six SoF tables implicitly. In eight tables the QoE was upgraded because of a large magnitude of effect; in one because of a dose-response association. In all nine tables risk of bias was the downgrading factor (unclear allocation concealment, incomplete outcome assessment, early stopping, no ITT analysis, outcome reporting bias), in one table in combination with imprecision and in two tables in combination with heterogeneity. Studying these SoF tables revealed that in all cases upgrading was not appropriate or at least questionable. In some SoF tables downgrading for imprecision was overlooked.
Conclusions: Upgrading of RCTs affected by study limitations was considered appropriate by Cochrane authors in 2% of the reviewed SoF tables, mostly because of a large magnitude of effect. According to the current GRADE guidance on upgrading, the upgrading was at least questionable. More detailed guidance and worked out examples are needed. These will be presented at the Colloquium.