Objectives: To assess the treatment of heterogeneity in systematic reviews of tests used to diagnose cancer in symptomatic patients.

Methods and results: A systematic search was used to identify relevant systematic reviews that reported an electronic search and compared test accuracy against a gold standard reference test.

The presence, source and treatment of heterogeneity within these reviews was assessed using a data sheet to extract details of: discussion of heterogeneity and its sources including methodological, threshold and statistical; methods to test for heterogeneity; and synthesis and reporting of results in the presence of heterogeneity.

We have found that many systematic reviews of diagnostic tests do not discuss heterogeneity of study results, even though many of these reviews include study results that appear heterogeneous. In reviews that do discuss heterogeneity, the presence and sources of heterogeneity is explored using a variety of techniques including formal statistical tests such as the chi-squared test, ROC curves, Galbraith plot and meta-regression.

Combining study results by meta-analysis without an assessment of heterogeneity, may lead to summary estimates of diagnostic test accuracy being overestimated or underestimated.

Conclusions: This survey highlights deficiencies in assessment and treatment of heterogeneity between studies included in systematic reviews for diagnostic tests in cancer. A new Cochrane handbook for systematic reviews of diagnostic tests is being written and will help guide future reviewers in the process of testing and summarising results in the presence of study result heterogeneity.

Acknowledgements: This project is funded by Cancer Research UK.