Objectives:
To describe, demonstrate and discuss three ways of performing statistical tests for sub-group differences, and whether changes should be made to written guidance and software available to Cochrane reviewers.
Background:
Version 5.0.0 of the Cochrane Handbook for Systematic Reviews of Interventions was published in February 2008. Section 9.6.3.1, “Is the effect different in different subgroups?” first explains a “simple approach” to testing for differences between subgroups based on partitioning the heterogeneity chi-square statistic Q. This method has been added to RevMan in version 5, so its use may well increase. The end of the section states that meta-regression provides “a more flexible alternative” that “may be regarded as preferable due to the high risk of false positive results when comparing subgroups in a fixed-effect model”. This is not currently available within RevMan.
Description:
The workshop will begin with an illustration each of following methods, using data from a Cochrane review published in 2006:
- The “simple approach” above, as used in the review (p=0.014)
- Random-effects meta-regression (p=0.205)
- An extension of Bucher”s method for adjusted indirect comparisons (Song et al. BMJ 2003;326:472) (p=0.662)
The workshop will then be opened to discussion around the following questions:
- Is the “simple approach” ever appropriate?
- If not, which is the best alternative?
- Would it be feasible to implement any of these in RevMan?
- If not, are there other ways of making these more widely available?
- Does section 9.6.3.1 of the Handbook need rephrasing?
A summary of the discussion will be made available to the Statistical Methods Group and the Handbook Advisory Group.