GRADE: reasons for downgrading the quality of evidence in recent Cochrane Reviews

Article type
Authors
Langendam M1, Hooft L1, Scholten R1
1Dutch Cochrane Centre, Amsterdam, Netherlands
Abstract
Background: The use of GRADE for evaluating the quality of evidence is recommended in latest version of the Cochrane Handbook (2008). GRADE specifies four quality levels (high, moderate, low and very low). Starting point is a high quality rating for a body of evidence based on randomized clinical trials. This rating can be downgraded, depending on the presence of five factors: within-study risk of bias, directness of evidence, heterogeneity, imprecision of the effect estimates and risk of publication bias. Objectives: To assess the use of GRADE in recently published Cochrane Reviews and to analyze reasons for downgrading. Methods: Our analysis will be based on all GRADE profiles in Cochrane Reviews published as new or updated reviews from 2008 until issue 6 2010. Results: In a pilot study, performed on the 33 new reviews in Issue 3, 2010, we found 4 GRADE profiles. In one review, the evidence for all outcomes was rated as high quality. The authors explained in several footnotes why they did not downgrade. In the other reviews the quality was downgraded: one review included a trial with results from a post hoc analysis of a sub-set of a larger trial and downgraded the quality of evidence because of reporting bias and indirectness. In the second review, with two trials and one outcome of interest, the quality of evidence was downgraded two levels because of poor study design, without further details. The confidence interval was wide (OR 0.61, 95% CI 0.09–4.27), but downgrading for imprecision was not mentioned. In the last review there was only one trial for each outcome. For each outcome, 4 to 5 reasons for downgrading were given, including ‘confidence interval includes the null’. Conclusions: GRADE is a methodologically rigorous and user friendly system, but in practice there is some inconsistency in applying the system.