Background: Screening and case finding for common mental health problems (such as depression) often relies on the use of brief paper and pencil psychometric instruments, which act as proxies for clinical diagnosis. As with all diagnostic technologies, there is an increasing interest in the application of systematic review techniques to summarise the key properties of these instruments (including the new Cochrane diagnostic reviews). To facilitate better reporting of diagnostic studies and the conduct of diagnostic reviews, two checklists have emerged: Statement for Reporting of Diagnostic Accuracy (STARD) and Quality Assessment of Diagnostic Accuracy Studies (QUADAS). To our knowledge, these instruments have not yet been applied in the evaluation of the diagnostic properties of psychometric instruments. Objectives: To examine the validity and usefulness of QUADAS when applied to diagnostic accuracy studies using psychometric instruments and to examine the quality in reporting of these studies during practical application of the checklist. Methods: Two reviewers independently rated the quality of 55 studies using QUADAS. Kappa statistics were calculated for each question to assess agreement between reviewers. Results: The overall agreement between the two reviewers for all QUADAS items combined was 85.6% (Kappa 0.71) indicating good agreement. The proportion of agreement between reviewers for each item ranged from just over 58% to 100% and was over 80% for eight of the items. The poorest agreement was associated with the items for selection criteria, indeterminate results and withdrawals. None of the studies adequately reported all relevant information to enable all QUADAS item to be scored as ‘yes’. Conclusions: Overall, QUADAS was relatively easy to use and appears to be an acceptable tool for appraising the quality of diagnostic accuracy studies using psychometric instruments. The application of QUADAS was hampered by the poor quality of reporting encountered.