Article type
Year
Abstract
Background: Studies of diagnostic test accuracy may be susceptible to a variety of biases due to deficiencies in their design, execution and analysis. Differences in methodological quality may partially explain high degrees of heterogeneity observed in diagnostic accuracy systematic reviews. Statistical investigations of heterogeneity aim to discern how aspects of methodological quality affect study findings. A review of the literature [1] identified that a broad range of statistical approaches are used to investigate sources of methodological heterogeneity, ranging in complexity from non-parametric tests for two group comparisons to hierarchical random effects regression models for binomial data. The focus of this study is on summary receiver operating characteristic (SROC), hierarchical SROC (HSROC) and bivariate regression models for the investigation of methodological heterogeneity.
Objectives: For sources of methodological heterogeneity, to assess empirically:
a. the importance of using SROC-based regression models over simple two-group comparisons, regression and meta-regression models;
b. whether the Moses SROC model (as implemented in diagnostic RevMan) is comparable to hierarchical and bivariate regression methods for investigating heterogeneity;
c. to investigate which approach within HSROC (accuracy-threshold-shape) and the bivariate normal (sensitivity-specificity) models for investigating heterogeneity is most suited to characterising heterogeneity.
Methods: Data are obtained from 31 meta-analyses, for which each study has been coded using 9 items from the QUADAS checklist. Whether each quality item is a significant source of heterogeneity was tested using each of the following 12 statistical methods for detecting heterogeneity (differences in sensitivity, specificity and diagnostic odds ratios tested using Mann-Whitney U-tests, t-tests, and meta-regression; Moses SROC method, HSROC model and the Bivariate Normal method). The consistency of the significance and estimates of effects was compared between methods.
Results: Will be presented at the Colloquium
Conclusions: Conclusions will inform the Cochrane Reviews of Diagnostic Test Accuracy Working Group on the reliability of investigations of heterogeneity using the Moses method that has been chosen to be programmed in the software.
1. Dinnes J, Deeks J, Kirby J, Roderick P. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. Health Technol Assess. 2005;9(12).
Objectives: For sources of methodological heterogeneity, to assess empirically:
a. the importance of using SROC-based regression models over simple two-group comparisons, regression and meta-regression models;
b. whether the Moses SROC model (as implemented in diagnostic RevMan) is comparable to hierarchical and bivariate regression methods for investigating heterogeneity;
c. to investigate which approach within HSROC (accuracy-threshold-shape) and the bivariate normal (sensitivity-specificity) models for investigating heterogeneity is most suited to characterising heterogeneity.
Methods: Data are obtained from 31 meta-analyses, for which each study has been coded using 9 items from the QUADAS checklist. Whether each quality item is a significant source of heterogeneity was tested using each of the following 12 statistical methods for detecting heterogeneity (differences in sensitivity, specificity and diagnostic odds ratios tested using Mann-Whitney U-tests, t-tests, and meta-regression; Moses SROC method, HSROC model and the Bivariate Normal method). The consistency of the significance and estimates of effects was compared between methods.
Results: Will be presented at the Colloquium
Conclusions: Conclusions will inform the Cochrane Reviews of Diagnostic Test Accuracy Working Group on the reliability of investigations of heterogeneity using the Moses method that has been chosen to be programmed in the software.
1. Dinnes J, Deeks J, Kirby J, Roderick P. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. Health Technol Assess. 2005;9(12).