Article type
Year
Abstract
Background: The sensitivity and specificity of a diagnostic test are often assumed to be independent of prevalence, that is the proportion of the study group with the disease of interest. Yet several studies and systematic reviews have reported differences in sensitivity and specificity related to prevalence.
Objectives: To explore the mechanisms that may that may be responsible for diagnostic accuracy varying with prevalence.
Methods: Identification and exploration of real and artefactual reasons why diagnostic accuracy may vary with disease prevalence, illustrated by examples from the literature.
Results: Factors responsible for differences in prevalence between studies or study subgroups can also be responsible for differences in sensitivity and specificity. Real variability is usually associated with spectrum effects:
o Patient spectrum itself: higher prevalence often results in a more severely diseased population in which the test performs better;
o Referral filter: inclusion depends on previous testing; these influence both prevalence and diagnostic accuracy;
o Reader expectation: diagnostic accuracy of readers can be influenced by the (supposed) prevalence in the study group. Artefactual variability can result from study design features:
o Additional exclusion criteria: if patients that are more difficult to classify are excluded, the test will seem to have a higher diagnostic accuracy;
o Verification bias: partial or differential verification leads to differences in prevalence as well as differences in diagnostic accuracy;
o Reference standard misclassification: when the reference standard is fallible, sensitivity will be less and specificity will be more underestimated as prevalence increases.
Conclusions: Sensitivity and specificity may vary with prevalence through several mechanisms. Differences in prevalence between studies can therefore act as a flag for differences in study population or study design. We encourage authors of systematic reviews to explore associations between prevalence and accuracy. We hope that more future systematic reviews will analyze and report such associations, and provide helpful explanations for these patterns, if they find them.
Objectives: To explore the mechanisms that may that may be responsible for diagnostic accuracy varying with prevalence.
Methods: Identification and exploration of real and artefactual reasons why diagnostic accuracy may vary with disease prevalence, illustrated by examples from the literature.
Results: Factors responsible for differences in prevalence between studies or study subgroups can also be responsible for differences in sensitivity and specificity. Real variability is usually associated with spectrum effects:
o Patient spectrum itself: higher prevalence often results in a more severely diseased population in which the test performs better;
o Referral filter: inclusion depends on previous testing; these influence both prevalence and diagnostic accuracy;
o Reader expectation: diagnostic accuracy of readers can be influenced by the (supposed) prevalence in the study group. Artefactual variability can result from study design features:
o Additional exclusion criteria: if patients that are more difficult to classify are excluded, the test will seem to have a higher diagnostic accuracy;
o Verification bias: partial or differential verification leads to differences in prevalence as well as differences in diagnostic accuracy;
o Reference standard misclassification: when the reference standard is fallible, sensitivity will be less and specificity will be more underestimated as prevalence increases.
Conclusions: Sensitivity and specificity may vary with prevalence through several mechanisms. Differences in prevalence between studies can therefore act as a flag for differences in study population or study design. We encourage authors of systematic reviews to explore associations between prevalence and accuracy. We hope that more future systematic reviews will analyze and report such associations, and provide helpful explanations for these patterns, if they find them.