Article type
Year
Abstract
Background: The problem of how to deal with heterogeneity of results is ubiquitous in systematic reviews. Appropriate approaches include the use of a small number of pre-specified hypotheses, and the interaction test.
Objective: To determine the rigor with which recent systematic reviews of therapy for diabetes explore heterogeneity.
Methods: We searched Medline, EMBASE, and Cochrane Database of Systematic Reviews for systematic reviews, published in any language from January 2008 to May 2010, that, first, assessed effects of insulin and anti-diabetes drug therapies, both in single use or combination, for any type of diabetes mellitus; and, second, reported a meta-analysis of two or more randomized controlled trials (RCTs). Two methods trained reviewers, independently and in duplicate, conducted study screening and data extraction.Results: Of 46 reviews that proved eligible, 29 (59%) explored heterogeneity by subgroup analyses (n = 27, 59%) and/or meta-regression (n = 5, 11%), and 21 out of 29 reviews (72%) reported heterogeneity p<0.1 or I-square$\geq$50%. Of the 29 reviews exploring heterogeneity, 6 (20%) clearly pre-specified heterogeneity variables, and 9 (31%) clearly used the interaction test. Explorations of heterogeneity tests a median of 5 (Inter-quartile: 2-6) variables. Variables explored included patient characteristics (17 of 29, 59%), intervention characteristics (24 of 29, 83%), and applied risk of bias and study characteristics (e.g. duration of follow up) (11 of 29, 38%). Many of these characteristics were also used as factors for sensitivity analyses (data not shown). Two of 5 studies including meta-regression, each having 14 and 22 RCTs, also undertook multivariable analyses of 4 independent variables.\HI{ Conclusions:} Exploration of heterogeneity was poorly performed in systematic reviews of insulin and drug therapies for diabetes. In particular, systematic reviewers often used patient baseline characteristics for exploring heterogeneity, which may be problematic. Systematic reviewers also did not well differentiate subgroup analyses and sensitivity analyses.
Objective: To determine the rigor with which recent systematic reviews of therapy for diabetes explore heterogeneity.
Methods: We searched Medline, EMBASE, and Cochrane Database of Systematic Reviews for systematic reviews, published in any language from January 2008 to May 2010, that, first, assessed effects of insulin and anti-diabetes drug therapies, both in single use or combination, for any type of diabetes mellitus; and, second, reported a meta-analysis of two or more randomized controlled trials (RCTs). Two methods trained reviewers, independently and in duplicate, conducted study screening and data extraction.Results: Of 46 reviews that proved eligible, 29 (59%) explored heterogeneity by subgroup analyses (n = 27, 59%) and/or meta-regression (n = 5, 11%), and 21 out of 29 reviews (72%) reported heterogeneity p<0.1 or I-square$\geq$50%. Of the 29 reviews exploring heterogeneity, 6 (20%) clearly pre-specified heterogeneity variables, and 9 (31%) clearly used the interaction test. Explorations of heterogeneity tests a median of 5 (Inter-quartile: 2-6) variables. Variables explored included patient characteristics (17 of 29, 59%), intervention characteristics (24 of 29, 83%), and applied risk of bias and study characteristics (e.g. duration of follow up) (11 of 29, 38%). Many of these characteristics were also used as factors for sensitivity analyses (data not shown). Two of 5 studies including meta-regression, each having 14 and 22 RCTs, also undertook multivariable analyses of 4 independent variables.\HI{ Conclusions:} Exploration of heterogeneity was poorly performed in systematic reviews of insulin and drug therapies for diabetes. In particular, systematic reviewers often used patient baseline characteristics for exploring heterogeneity, which may be problematic. Systematic reviewers also did not well differentiate subgroup analyses and sensitivity analyses.