Background: Publication bias is an issue for meta-analyses of test accuracy, as it is for randomised trials, but it is questionable whether the same funnel plots and tests of asymmetry can be used to detect bias.

Objectives: To investigate the performance of existing and new tests of funnel plot asymmetry for systematic reviews of diagnostic test accuracy.

Methods: Multiple diagnostic test accuracy meta-analyses with varying sample sizes, disease prevalence, test threshold, diagnostic accuracy, and relationships between diagnostic accuracy and threshold were created by simulation. Funnel plots of log diagnostic odds ratios against standard error were produced for meta-analyses created with and without publication bias, and asymmetry tested using the Egger, Begg and Macaskill tests. An alternative funnel plot of log diagnostic odds ratio against the square root of effective sample size was also considered, and associated regression and correlation tests of asymmetry performed. Each meta-analysis was simulated 10,000 times, and Type I and Type II error rates assessed for each test.

Results: The Egger, Begg and Macaskill tests had inflated Type I error rates for meta-analyses where diagnostic odds ratios were high, the prevalence of disease differed from 50%, and thresholds favoured sensitivity over specificity, or vice versa, all of which are typical characteristics of meta-analyses of diagnostic test accuracy. Tests based on functions of effective sample size were valid if occasionally conservative tests for publication bias.

Conclusions: Existing funnel plots and tests for bias that utilise standard errors of odds ratios are likely to be seriously misleading if applied to meta-analyses of diagnostic test accuracy. Funnel plots using functions of effective sample size and associated regression tests of asymetry should be used instead.

Acknowledgements: This work was undertaken by the STEP programme based at the University of Sydney funded by NHMRC grant no 211205.