Article type
Year
Abstract
Background:
Meta-analysis may produce estimates that are unrepresentative of a test’s performance in practice. Tailored meta-analysis circumvents this by deriving an applicable region for the practice and selecting the studies compatible with the region. It requires the test positive rate, r, and prevalence, p, to be estimated for the setting; previous studies have assumed their independence.
Objective:
To investigate the effect that a correlation between test positive rate and prevalence has on estimating the applicable region, and how this affects tailored meta-analysis.
Methods:
We investigated four methods for estimating 99% confidence intervals for r and p: Wilson’s score, Clopper-Pearson’s exact interval, the Bonferroni correction and Hotelling’s T2 statistic. We analysed these in terms of the coverage probability using simulation trials over different correlations, sample sizes, and values for r and p. We then applied the methods to two published meta-analyses with associated practice data and evaluated the effects on the applicable region, studies selected and summary estimates.
Results:
Hotelling’s T2 statistic with a continuity correction had the highest median coverage (0.9971). This and the Clopper-Pearson method with a Bonferroni correction both had coverage consistently above 0.99. The coverage of Hotelling’s T2 statistic intervals varied the least across different correlations. For both meta-analyses, the number of studies selected was largest when Hotelling’s T2 statistic was used to derive the applicable region. In one instance this increased the sensitivity by over 4% compared with tailored meta-analysis estimates using other methods.
Conclusion:
Tailored meta-analysis returns estimates that are tailored to practice, providing the applicable region is accurately defined. This is most likely when the 99% confidence interval for test positive rate and prevalence are estimated using Hotelling’s T2 statistic with a continuity correction. Potentially, the applicable region may be obtained using routine electronic health data.
Meta-analysis may produce estimates that are unrepresentative of a test’s performance in practice. Tailored meta-analysis circumvents this by deriving an applicable region for the practice and selecting the studies compatible with the region. It requires the test positive rate, r, and prevalence, p, to be estimated for the setting; previous studies have assumed their independence.
Objective:
To investigate the effect that a correlation between test positive rate and prevalence has on estimating the applicable region, and how this affects tailored meta-analysis.
Methods:
We investigated four methods for estimating 99% confidence intervals for r and p: Wilson’s score, Clopper-Pearson’s exact interval, the Bonferroni correction and Hotelling’s T2 statistic. We analysed these in terms of the coverage probability using simulation trials over different correlations, sample sizes, and values for r and p. We then applied the methods to two published meta-analyses with associated practice data and evaluated the effects on the applicable region, studies selected and summary estimates.
Results:
Hotelling’s T2 statistic with a continuity correction had the highest median coverage (0.9971). This and the Clopper-Pearson method with a Bonferroni correction both had coverage consistently above 0.99. The coverage of Hotelling’s T2 statistic intervals varied the least across different correlations. For both meta-analyses, the number of studies selected was largest when Hotelling’s T2 statistic was used to derive the applicable region. In one instance this increased the sensitivity by over 4% compared with tailored meta-analysis estimates using other methods.
Conclusion:
Tailored meta-analysis returns estimates that are tailored to practice, providing the applicable region is accurately defined. This is most likely when the 99% confidence interval for test positive rate and prevalence are estimated using Hotelling’s T2 statistic with a continuity correction. Potentially, the applicable region may be obtained using routine electronic health data.