Article type
Year
Abstract
Background: Meta-analysis (MA) is frequently used to synthesize disparate results of diagnostic test studies. With diagnostic test quality becoming a focus of the medical errors literature, it is an appropriate time to assess the current state of meta-analysis for diagnostic test studies.
Objective: To collect and describe all meta-analyses of diagnostic tests published in the peer-reviewed literature in order to assess their quality, analytic methods and potential for examining heterogeneity. Topics included sources and languages searched, search strategies, type of test, adherence to the STARD quality standards of design, appropriateness of statistical methods and availability of covariates for exploring heterogeneity. Methods: We conducted a MEDLINE search of the literature through mid-2003, supplemented by examining references of relevant methods and review articles, to find all meta-analyses of diagnostic tests defined as any test with the purpose of detecting or predicting a target medical condition. The meta-analysis must have provided a quantitative summary of one or more statistics relating to test performance. Systematic reviews not providing such a summary were excluded as were studies that evaluated the use of diagnostic tests as an intervention (e.g., screening mammography). The authors screened abstracts identified by a broad search strategy and extracted data from retrieved papers that satisfied entry criteria.
Results: We retrieved 236 MAs, of which 6 were published before 1990, 25 between 1990-1994, 93 from 1995-1999 and 112 since 2000. 44% involved imaging tests, 28% laboratory tests and 13% clinical examinations. The number of studies cited in each article ranged between 2 and 100 with a median of 19. Many summary reports reported on multiple MAs. Most of the MAs described a search strategy with the most common sources searched being: Medline (90%), bibliographies (73%), reviews (26%), experts (20%) and Embase (17%). Fifteen percent used unpublished sources and the number using Cochrane has increased to 10% since 2000. Only 14% reported searching all languages, whereas 40% searched only English language articles and 35% did not report the languages searched. Search words have become more commonly included. Whereas older articles tended to use fixed effects methods and simple pooling, more recent studies have increasingly used random effects models (35% since 2000). Sensitivity and specificity continue to be summarized in nearly 65% of studies, but the summary receiving operating characteristic (SROC) curve is now used in 47% of MAs. Regression is still underutilized, only 18% of the time, even though covariates are reported 68% of the time. Quality of reporting has improved with the nature of enrollment (consecutive or random) increasing from 16% to 37% and blinding reported 54% compared with 42% before 1990.
Conclusions: Meta-analyses of diagnostic tests continue to increase from a handful ten years ago to more than 30 per year recently. Published standards have improved their quality, but overall quality remains low. Despite the introduction of the SROC and random effects methods, few studies use regression to explore heterogeneity; many studies still use pooling and fixed effect methods or ignore heterogeneity altogether. Such inadequate methods may invalidate study findings.
Objective: To collect and describe all meta-analyses of diagnostic tests published in the peer-reviewed literature in order to assess their quality, analytic methods and potential for examining heterogeneity. Topics included sources and languages searched, search strategies, type of test, adherence to the STARD quality standards of design, appropriateness of statistical methods and availability of covariates for exploring heterogeneity. Methods: We conducted a MEDLINE search of the literature through mid-2003, supplemented by examining references of relevant methods and review articles, to find all meta-analyses of diagnostic tests defined as any test with the purpose of detecting or predicting a target medical condition. The meta-analysis must have provided a quantitative summary of one or more statistics relating to test performance. Systematic reviews not providing such a summary were excluded as were studies that evaluated the use of diagnostic tests as an intervention (e.g., screening mammography). The authors screened abstracts identified by a broad search strategy and extracted data from retrieved papers that satisfied entry criteria.
Results: We retrieved 236 MAs, of which 6 were published before 1990, 25 between 1990-1994, 93 from 1995-1999 and 112 since 2000. 44% involved imaging tests, 28% laboratory tests and 13% clinical examinations. The number of studies cited in each article ranged between 2 and 100 with a median of 19. Many summary reports reported on multiple MAs. Most of the MAs described a search strategy with the most common sources searched being: Medline (90%), bibliographies (73%), reviews (26%), experts (20%) and Embase (17%). Fifteen percent used unpublished sources and the number using Cochrane has increased to 10% since 2000. Only 14% reported searching all languages, whereas 40% searched only English language articles and 35% did not report the languages searched. Search words have become more commonly included. Whereas older articles tended to use fixed effects methods and simple pooling, more recent studies have increasingly used random effects models (35% since 2000). Sensitivity and specificity continue to be summarized in nearly 65% of studies, but the summary receiving operating characteristic (SROC) curve is now used in 47% of MAs. Regression is still underutilized, only 18% of the time, even though covariates are reported 68% of the time. Quality of reporting has improved with the nature of enrollment (consecutive or random) increasing from 16% to 37% and blinding reported 54% compared with 42% before 1990.
Conclusions: Meta-analyses of diagnostic tests continue to increase from a handful ten years ago to more than 30 per year recently. Published standards have improved their quality, but overall quality remains low. Despite the introduction of the SROC and random effects methods, few studies use regression to explore heterogeneity; many studies still use pooling and fixed effect methods or ignore heterogeneity altogether. Such inadequate methods may invalidate study findings.