Article type
Year
Abstract
Background: An essential part of a Cochrane overview is the assessment of the quality of evidence within included reviews. We planned to use the GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach in our overview (as recommended in the Cochrane Handbook), but its subjectivity led to inconsistency of application.
Objectives: To develop and use an algorithm to assign GRADE levels of evidence objectively to comparisons within reviews included in a Cochrane Overview.
Methods: After initial exploration of applying GRADE levels of evidence, authors agreed criteria perceived to be most relevant for judging quality of the particular evidence synthesised within the overview. Key criteria judged to be of most importance were: number of participants; risk of bias of trials, heterogeneity; and methodological quality of the review. An initial algorithm was drafted, applied to a convenience sample of 43 comparisons, and compared to previous, independently-applied, subjective judgements of GRADE level. An iterative process explored impact of criteria 'weighting' within the algorithm and the number of consequent downgrades of quality level. We created and applied four different formulae to assign downgrades to each of the 43 sample comparisons and explored the resulting levels of evidence.
Results: An algorithm, judged to assign the most appropriate levels of evidence (see Table 1), and a formula, for assigning GRADE level of evidence based on number of downgrades determined using the algorithm (see Table 2), were developed. This algorithm was applied to all 127 comparisons included within the overview.
Conclusions: This algorithm enabled us to justify and report the GRADE level of evidence clearly for each comparison, and also to be consistent, transparent and efficient in decisions around levels of evidence for each comparison. Whilst mechanistic in application, this algorithm aims to capture what was subjectively judged to be of greatest importance to the quality of this specific evidence-base. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.
Objectives: To develop and use an algorithm to assign GRADE levels of evidence objectively to comparisons within reviews included in a Cochrane Overview.
Methods: After initial exploration of applying GRADE levels of evidence, authors agreed criteria perceived to be most relevant for judging quality of the particular evidence synthesised within the overview. Key criteria judged to be of most importance were: number of participants; risk of bias of trials, heterogeneity; and methodological quality of the review. An initial algorithm was drafted, applied to a convenience sample of 43 comparisons, and compared to previous, independently-applied, subjective judgements of GRADE level. An iterative process explored impact of criteria 'weighting' within the algorithm and the number of consequent downgrades of quality level. We created and applied four different formulae to assign downgrades to each of the 43 sample comparisons and explored the resulting levels of evidence.
Results: An algorithm, judged to assign the most appropriate levels of evidence (see Table 1), and a formula, for assigning GRADE level of evidence based on number of downgrades determined using the algorithm (see Table 2), were developed. This algorithm was applied to all 127 comparisons included within the overview.
Conclusions: This algorithm enabled us to justify and report the GRADE level of evidence clearly for each comparison, and also to be consistent, transparent and efficient in decisions around levels of evidence for each comparison. Whilst mechanistic in application, this algorithm aims to capture what was subjectively judged to be of greatest importance to the quality of this specific evidence-base. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.