Sensitivity and specificity rarely vary with prevalence

Article type
Authors
Leeflang M1, Hooft L2, Reitsma J1, Bossuyt P1
1Department of Clinical Epidemiology, Biostatistics and Bioinformatics, University of Amsterdam
2Dutch Cochrane Centre, Amsterdam
Abstract
Background: Anecdotal evidence shows that sensitivity and specificity tend to vary with prevalence, but empirical studies systematically examining this relationship are not available. Objective: To investigate the impact of prevalence on sensitivity and specificity in a large set of meta-analyses of diagnostic accuracy studies. Methods: From a set of 30 meta-analyses, containing 487 diagnostic accuracy studies, we selected meta-analyses with at least 10 studies that had included a consecutive series of eligible patients and in which the prevalence between studies varied at least 20 percentage points. We obtained summary estimates of sensitivity and specificity for each review, using the bivariate logitnormal method for meta-analysis, including prevalence as a covariate to investigate its effect on sensitivity and specificity. Results: Twenty-one reviews fulfilled our criteria. They contained between 10 and 39 studies, with a range in prevalence between 20 and 78 percentage points. Overall, prevalence had a significant effect on both sensitivity and specificity. However, when analyzing the reviews separately, prevalence had a significant effect on both sensitivity and specificity in only two reviews. In four reviews, prevalence had a significant effect on specificity only. In none of the reviews prevalence had a significant effect on sensitivity only. Conclusions: Diagnostic accuracy may change with varying prevalence, but it is not common to find significant effects on accuracy statistics in individual systematic reviews. This may in part be due to the limited power of meta-regression to explain heterogeneity with study level covariates, aggravated by the generally low prevalence (median 35%; 25th percentile 15%). Because the main underlying mechanism how prevalence may alter diagnostic accuracy are differences in disease spectrum and case-mix, we will further investigate these relationships as well and present those data during the Colloquium.