Rating quality of evidence using Cochrane/GRADE approach: sub-group analysis

Article type
Authors
Guyatt G, Alonso-Coello P, Oxman A, Kunz R, Helfand M, Brozek J
Abstract
Background: The Cochrane Handbook suggests that Cochrane reviews use the GRADE approach to rate quality of evidence (degree of confidence in the estimates of effect). Review authors face challenges in applying this approach to the exploration of heterogeneity and potential sub-group effects. Criteria: Authors must judge the credibility of the putative sub-group effect. The Cochrane Handbook suggests seven criteria: within-versus between-group comparisons; likelihood of chance explaining the difference; consistency across studies; a priori specification of the sub-group hypothesis; the number of a priori hypotheses; the size of the difference in effect between sub-groups; and biological plausibility. If the sub-group effect is judged implausible reviewers should present the effect for the total group. If the sub-group effect is judged highly credible, authors should present the magnitude of effect for each sub-group. In neither of these situations is rating down for inconsistency necessary. When sub-group effects are uncertain, judgment is required. If authors conclude that a sub-group effect is unlikely, but still possible, they are likely to present the effect for the total group only, but rate down because of inconsistency. If they judge the sub-group effect likely but still uncertain, authors may present the magnitude of effect for each sub-group. They may rate down for inconsistency in one or both of the sub-groups because remaining uncertainty about the credibility of the sub-group effect decreases confidence in the estimates of effect from either sub-group. Application: The presentation will apply the criteria to an example of a systematic review of vitamin D and calcium for fracture prevention in which the sub-group hypothesis is that the effect differs according to age. The example will illustrate the challenges because the evidence for the sub-group effect meets some but not all criteria, and is thus equivocal.