Assessing the methodological quality of systematic review: the AMSTAR II-DTA extension

Tags: Oral
Cinquini M1, Moschetti I1, Minozzi S2
1Istituto di Ricerche Farmacologiche Mario Negri IRCCS, 2Department of Epidemiology, Lazio Regional Health Service

Background: in 2017, the AMSTAR II checklist was proposed as an improvement of the previous AMSTAR to assess the methodological quality of systematic reviews (SR) that include randomized or non-randomized studies of healthcare interventions. As for SRs of interventions, SRs of diagnostic accuracy (DTA-SR) are used widely in production of guidelines, and often more than one SR is found for a specific topic. So, assessing their methodological quality is crucial. Although AMSTAR II could be used to evaluate DTA-SR, it lacks specific topics regarding this type of SR that raises some concerns.

Objectives: to present a possible extension of AMSTAR II for assessing the methodological quality of DTA-SR

Methods: we used a convenience sample of reviews on the usefulness of radiopharmaceuticals for diagnosis of neuroendocrine tumours, evaluated in a European project for the development of clinical guidelines.

Starting from the existing tool, we tried to revise some of the question to be better suited to DTA.

Results: based on a preliminary evaluation of 17 systematic reviews of diagnostic accuracy, we propose a revised version of questions 1, 3, 8, 9, and 11.

- In question 1, we focused attention on the index text and reference standard to achieve a Partial Yes answer and the purpose of the SR, that is, triage, replacement, add-on to achieve a Yes answer.

- In question 3, we modified the type of study to be included in the reference standard considering non-experimental, cross-sectional (case-control accuracy or cohort accuracy type) or randomized controlled trials in which people were randomized to the index and comparator test(s) and all received the reference standard test.

- We modified questions 8 and 9 considering specific details of the primary study to be reported, such as details of population/target condition, details of index test and reference test (including procedures for diagnosis), details of the timeframe (including timing of assessment/delay between tests) to achieve a Yes, and the QUADAS-II tool to assess risk of bias, respectively.

- Question 11 was the most difficult to be tailored on SR-DTA, due to the several types of study designs to include.

We have integrated all the changes in Table 1.

Conclusions: this is the first draft of a more specific tool for the methodological evaluation of DTA-SR. We recognize that the tool can be improved and piloted on a greater number of systematic reviews to better adapt it to the specific context. We will present the results of its assessment against a greater number of DTA reviews.

Patient or healthcare consumer involvement: the project focuses on methods to assess methodological quality, so we could not involve consumers.