Article type
Year
Abstract
Background: Cochrane has adopted GRADE’s approach (Grading of Recommendations, Assessment, Development and Evaluations) to categorizing quality or certainty of evidence, defined as reflecting the extent of confidence that the estimates of effects are correct. This definition leaves ambiguity: does it refer to confidence in point estimates, or confidence in the range of possible estimates?
Objectives: To clarify the definition and approaches to judging certainty of evidence.
Methods: Brainstorming, workshops, presentations, iterative refinement of ideas, and discussion at two GRADE Working Group meetings.
Results: Quality of evidence is best considered as the confidence or certainty that a true effect lies on one side of a specified threshold, or within a chosen range.
This interpretation raises the challenge of defining the threshold or range for making quality ratings. We developed four possible approaches for making this judgment. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. We defined three less contextualized approaches more appropriate for systematic reviews (Table).
The three approaches most appropriate for systematic reviews involve judging certainty that the true effect lies within the 95% confidence interval; the effect is something other than no effect at all; and that the effect lies within ranges of what we might consider small, moderate, or large.
We have applied the approaches to real-life systematic reviews and will present this application at the Colloquium.
Conclusions: The Grade Working Group is now considering this novel characterization of certainty of evidence, the implications for guidelines and systematic reviews. If adopted by GRADE, and subsequently by Cochrane, it will provide a useful clarification of how Cochrane reviewers can make certainty of evidence judgments.
Objectives: To clarify the definition and approaches to judging certainty of evidence.
Methods: Brainstorming, workshops, presentations, iterative refinement of ideas, and discussion at two GRADE Working Group meetings.
Results: Quality of evidence is best considered as the confidence or certainty that a true effect lies on one side of a specified threshold, or within a chosen range.
This interpretation raises the challenge of defining the threshold or range for making quality ratings. We developed four possible approaches for making this judgment. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. We defined three less contextualized approaches more appropriate for systematic reviews (Table).
The three approaches most appropriate for systematic reviews involve judging certainty that the true effect lies within the 95% confidence interval; the effect is something other than no effect at all; and that the effect lies within ranges of what we might consider small, moderate, or large.
We have applied the approaches to real-life systematic reviews and will present this application at the Colloquium.
Conclusions: The Grade Working Group is now considering this novel characterization of certainty of evidence, the implications for guidelines and systematic reviews. If adopted by GRADE, and subsequently by Cochrane, it will provide a useful clarification of how Cochrane reviewers can make certainty of evidence judgments.