An unusual diagnostic test accuracy review: challenges in study identification, data extraction and quality assessment

Article type
Authors
Probyn K1, Sguassero Y1, Smith R2, Leeflang MM3, Bergman H1, Buckley B1, Chaplin M4, Cogo E1, Cohen K5, Ford N6, Henschke N1, Orrell C7, Petkovic J1, Sebastianski M1, Garner P8, Hine P9, Villanueva G1
1Cochrane Response, Development Directorate, Cochrane Central Executive Team
2Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town
3Department of Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam
4Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool
5Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town
6Department of HIV & Global Hepatitis Programme, World Health Organization, Geneva
7Desmond Tutu HIV Centre, University of Cape Town, Cape Town
8Professor emeritus, Department of Clinical Sciences, Liverpool School of Tropical Medicine
9Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool
Abstract
Background:
This was a joint project of Cochrane Response and Cochrane Infectious Diseases Group. The aim of the review was to determine the accuracy of simple measures of antiretroviral therapy (ART) adherence for detecting non-suppressed viral load in people with HIV that could be utilized in resource-limited settings. Included studies measured adherence to ART in different ways and compared the level of adherence to viral non-suppression.

Objectives:
To describe the methodological challenges of conducting a diagnostic test accuracy (DTA) systematic review with studies that were not designed to evaluate DTA.

Methods:
The index tests were any measures of ART adherence that could be utilized in resource‐limited settings (self-report, tablet counts, pharmacy records or electronic monitoring). The reference standard was a non-suppressed viral load, as detected using nucleic acid testing technologies. We extracted the 2×2 data using pre-piloted extraction forms. We assessed the methodological quality using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, which was tailored to fit the review question.

Results:
We included 51 studies. Most studies were not set out to assess the measure of adherence as a diagnostic test but used other study designs also reporting on antiretroviral adherence. By agreeing on a set of “rules” on how to apply QUADAS-2, we were able to achieve consistency between reviewers. We encountered high variability for all index tests, and the overall certainty of evidence was very low. No measure offered either a sufficiently high sensitivity or specificity to detect viral non‐suppression.

Conclusions:
Identifying studies, data extraction and quality assessment were challenging because the studies we included were not designed as diagnostic accuracy studies. However, we successfully managed to include these studies by tailoring the data extraction form and the QUADAS-2 signaling questions.

Patient, public and/or healthcare consumer involvement:
There was no direct patient involvement in this review. However, the results of this review are very relevant for people with HIV living in resource-limited settings where viral load testing is not universally available or not feasible. The interim results of this review informed the new World Health Organization (WHO) HIV guideline.