Partial and differential verification in diagnostic accuracy studies

Article type
Authors
WS Rutjes A, B Reitsma J, Irwig L, MM Bossuyt P
Abstract
Background: In diagnostic accuracy studies, index test results are verified by comparing them with the results of the reference standard. In some studies there is partial verification, when not all those receiving the index test proceed to the reference standard, or differential verification, when more than one reference standards is used. Objectives and Methods: To explore the mechanisms that can lead to changes in accuracy estimates in studies with partial or differential verification. Methods used were theoretical reasoning and meta-epidemiologic modelling. Results: Incomplete verification with the preferred reference standard may occur by design, but more commonly occurs because the practitioner or patient decides to avoid verification by an invasive test. Incomplete verification is more likely to occur in patients in whom the index test is negative and in patients with a low prior probability of disease. Although our meta-epidemiologic approach could not demonstrate average changes in estimates of diagnostic accuracy due to partial verification, omission of unverified index test is expected to lead to overestimation of sensitivity and underestimation of specificity. Studies that use differential verification report significantly higher estimates of specificity (1.4, 95% CI: 1.0 to 1.9) and odds ratio (1.8, 95% CI: 1.0 to 3.1) compared to studies using a single reference standard in the verification of test results. These changes depend on both the associations between the index test and the reference standards and the proportions of people that are verified differently. Conclusions: When designing a diagnostic accuracy study, researchers should avoid partial and differential verification. If that is impossible and different reference standards are used, the best approach is to design the study so that all patients positive on the index test are verified by the one reference standard and all patients negative on the index test are verified by the other reference standard. The appropriate measures of accuracy are then the positive and negative predictive values with the specified reference standards. When that is not done, data should be provided separately for each index test-reference standard combination.