Article type
Year
Abstract
Background: Selection of medical tests is critical to health technology assessment. For comparing summary sensitivities and specificities (summary points) of competing tests, Cochrane recommends bivariate meta-regression models. However, fitting these complex models is sometimes challenging and simpler alternatives are needed in such situations.
Objectives: To assess the performance of univariate and bivariate random-effects logistic meta-regression models for comparing diagnostic accuracy, and to examine the effect of different variance-covariance structures on each model.
Methods: Systematic reviews and meta-analyses of test accuracy in the Database of Abstracts of Reviews of Effects published between 1994 and October 2012 were identified. Univariate and bivariate models with different variance-covariance specifications were fitted to meta-analytic datasets from the reviews. We compared summary estimates from the models in terms of differences in magnitude, precision, statistical significance and direction of effect (i.e. qualitative change in test ranking).
Results: We included 57 reviews that evaluated the accuracy of two tests and provided data for comparative meta-analyses. Across 48 test comparisons where both univariate and bivariate models converged, differences in magnitude and precision of relative sensitivities and relative specificities were negligible. With univariate models as the reference, median (interquartile range) ratios of relative sensitivities and relative specificities were 1.00 (1.00 to 1.01) and 1.00 (1.00 to 1.00). In contrast, important differences such as changes in statistical significance and test rankings were often observed between findings from univariate or bivariate models with different variance-covariance structures.
Conclusions: Simplifying bivariate meta-regression models to univariate models is likely to be a valid alternative when estimation problems are encountered in a comparative meta-analysis. However, joint inferences cannot be made about sensitivity and specificity. If data permits, assumptions about variance-covariance structures should be checked when fitting the models.
Objectives: To assess the performance of univariate and bivariate random-effects logistic meta-regression models for comparing diagnostic accuracy, and to examine the effect of different variance-covariance structures on each model.
Methods: Systematic reviews and meta-analyses of test accuracy in the Database of Abstracts of Reviews of Effects published between 1994 and October 2012 were identified. Univariate and bivariate models with different variance-covariance specifications were fitted to meta-analytic datasets from the reviews. We compared summary estimates from the models in terms of differences in magnitude, precision, statistical significance and direction of effect (i.e. qualitative change in test ranking).
Results: We included 57 reviews that evaluated the accuracy of two tests and provided data for comparative meta-analyses. Across 48 test comparisons where both univariate and bivariate models converged, differences in magnitude and precision of relative sensitivities and relative specificities were negligible. With univariate models as the reference, median (interquartile range) ratios of relative sensitivities and relative specificities were 1.00 (1.00 to 1.01) and 1.00 (1.00 to 1.00). In contrast, important differences such as changes in statistical significance and test rankings were often observed between findings from univariate or bivariate models with different variance-covariance structures.
Conclusions: Simplifying bivariate meta-regression models to univariate models is likely to be a valid alternative when estimation problems are encountered in a comparative meta-analysis. However, joint inferences cannot be made about sensitivity and specificity. If data permits, assumptions about variance-covariance structures should be checked when fitting the models.