Article type
Year
Abstract
Background: The decision on whether a diagnostic test accuracy study is applicable to practice is largely a qualitative process. Consequently, the summary estimates provided by meta-analysis may be, in some cases, completely inaccurate for a particular setting.
Objectives: To develop a new method in which the selection of applicable studies is based on more quantitative criteria.
Methods: It is shown how routine data collected on the test positive rate and prevalence from the setting of interest may be used to define an ‘applicable region’ for studies in ROC space. Studies are selected based on both qualitative criteria and the probability that their study estimate for the false positive rate and sensitivity arose from their parameters lying in the applicable region. Three methods for calculating these probabilities are developed and used to tailor the selection of studies for meta-analysis. The Pap test applied to the NHS cervical screening programme provides a case example.
Results: The original meta-analysis included 68 studies. In contrast, tailoring the selection using NHS data resulted in at most 17 studies being considered plausible for the NHS. From conventional meta-analysis the sensitivity and specificity for the Pap test were estimated to be 72.8% (95% CI:65.8–78.8) and 75.4% (95% CI:68.1–81.5) compared with 50.9% (35.8–66.0) and 98.0% (95% CI: 95.4–99.1) from tailored meta-analysis using a binomial method for selection. The positive likelihood ratio increased from 3.0 (95% CI:2.4–3.7) to 25.6 (95%:10.1–65.0) between the conventional and tailored meta-analysis. Thus, for a background prevalence for CIN 1 of 2.2%, the post-test probability for CIN 1 increases from 6.2 to 36.6%.
Conclusions: Tailored meta-analysis provides a method for augmenting study selection based on their applicability to a setting. As such the summary estimate is more likely to be plausible for a setting and could improve diagnostic prediction in practice.
Objectives: To develop a new method in which the selection of applicable studies is based on more quantitative criteria.
Methods: It is shown how routine data collected on the test positive rate and prevalence from the setting of interest may be used to define an ‘applicable region’ for studies in ROC space. Studies are selected based on both qualitative criteria and the probability that their study estimate for the false positive rate and sensitivity arose from their parameters lying in the applicable region. Three methods for calculating these probabilities are developed and used to tailor the selection of studies for meta-analysis. The Pap test applied to the NHS cervical screening programme provides a case example.
Results: The original meta-analysis included 68 studies. In contrast, tailoring the selection using NHS data resulted in at most 17 studies being considered plausible for the NHS. From conventional meta-analysis the sensitivity and specificity for the Pap test were estimated to be 72.8% (95% CI:65.8–78.8) and 75.4% (95% CI:68.1–81.5) compared with 50.9% (35.8–66.0) and 98.0% (95% CI: 95.4–99.1) from tailored meta-analysis using a binomial method for selection. The positive likelihood ratio increased from 3.0 (95% CI:2.4–3.7) to 25.6 (95%:10.1–65.0) between the conventional and tailored meta-analysis. Thus, for a background prevalence for CIN 1 of 2.2%, the post-test probability for CIN 1 increases from 6.2 to 36.6%.
Conclusions: Tailored meta-analysis provides a method for augmenting study selection based on their applicability to a setting. As such the summary estimate is more likely to be plausible for a setting and could improve diagnostic prediction in practice.