Using full information on cut-offs per study to determine an optimal cut-off in diagnostic test accuracy reviews

Article type
Authors
Rücker G1, Steinhauser S1, Schumacher M1
1Medical Center, University of Freiburg, Germany
Abstract
Background: In diagnostic test accuracy (DTA) reviews, it is often assumed that each study reports only one pair of specificity (Sp) and sensitivity (Se). The established bivariate model considers the joint distribution of Sp and Se across studies. However, in primary studies (Sp, Se) pairs are often reported for two or more cut-offs, and the cut-off values are reported as well.
Objectives: To use this additional information for modelling the distributions of the underlying biomarker for diseased and non-diseased individuals and to determine an optimal cut-off.
Methods: We assume that for some or all DTA studies in a meta-analysis a number of cut-offs is reported, each with corresponding estimates of Sp and Se. These provide direct information about the empirical cumulative distribution function (ecdf) for both groups of individuals. We propose a class of hierarchical models for the distribution including study as a fixed or random factor. These models lead to average ecdfs for both groups of individuals. As the difference between these is the Youden index as a function of the cut-off, an optimal cut-off can be found by maximising this function. A summary receiver operating characteristic (ROC) curve is estimated based on the distributions.
Results: The approach is demonstrated on three meta-analyses of alcohol screening, procalcitonin as a marker for sepsis and diagnosis of asthma.
Conclusions: If there are a number of studies reporting at least two cut-offs with (Sp, Se) per study, we can determine an optimal cut-off and estimate a summary ROC curve based on all available information from the primary studies.