The assessment of quality of meta-analyses in diagnostic research – a systematic review

Article type
Authors
Harvey-Willis B1
1Health Methodology, University of Manchester, UK
Abstract
Background: Over the past twenty years the number of meta-analyses in diagnostic research has increased dramatically and this has coincided with methodological and statistical developments in the field. The current state of quality or the effect the recent developments have had on quality has not been evaluated and motivates this review. Objective: To assess the quality of published meta-analyses of diagnostic test studies. Methods: The following databases were searched: MEDLINE, EMBASE, CINAHL, Cochrane, PsychInfo, HMIC, AMED, and Global Health. Studies were included if all of the following were satisfied: Evaluated a diagnostic test; measured test performance; searched two or more databases; stated the search terms and inclusion criteria; used a statistical method to summarise performance. Quality was evaluated on two levels. The review process was assessed by its compliance with PRISMA and by qualitative appraisal; the component primary studies were assessed by adapting the QUADAS tool. Data were extracted on the study characteristics, items in PRISMA and QUADAS. Fisher’s exact test and the Z test were used when making comparisons. Results: There were 237 studies included. Overall the quality of meta-analyses has improved and reviewers are increasingly assessing the quality of the component primary studies. In five PRISMA items there were statistically significant improvements between the periods 2001–2004 and 2005–2008. However, in many items including investigating heterogeneity, analyses were not performed in the majority of studies. Qualitative appraisal revealed insufficient detail to allow reproducibility of results and inadequate consideration towards ensuring completeness. The results of applying QUADAS demonstrated there were not only deficiencies in the reporting in the primary studies, but quality assessments by reviewers were also not fully reported. Conclusion: Although there has been an improvement in the quality of meta-analyses there are still many deficiencies in both the review process and reporting that future reviewers need to address.