Article type
Year
Abstract
Background: diagnostic accuracy studies frequently report multiple cut-off values, which poses a challenge for systematic review authors. The Cochrane Handbook for Systematic Reviews of Interventions currently recommends estimating summary measures for a single common cut-off value. With this strategy, information is lost by neglecting alternative cut-offs, and there may be risk of bias if the cut-off was data-driven. A new method allows consideration of multiple cut-offs from a single study, providing an alternative approach for meta-analysis.
Objectives: to compare the alternative multiple-cut-off strategy against the recommended single-cut-off strategy in meta-analysis of diagnostic accuracy studies.
Methods: we rely on two systematic reviews of biomarkers among adults with non-alcoholic fatty liver disease (NAFLD): cytokeratin-18 M30 fragment (CK-18) and the Enhanced Liver Fibrosis (ELF) score. We performed a comprehensive search in five databases to identify diagnostic accuracy studies of CK-18 and ELF in detecting non-alcoholic steatohepatitis or advanced fibrosis among adults with NAFLD. We conducted two separate meta-analyses per biomarker: the first included a single cut-off value, the second included all cut-off values reported in each study.
Results: we included a total of 18 studies in the meta-analysis for CK-18 and nine for ELF. Under the multiple-cut-off model, we considered 33 cut-off values for CK-18 and 16 for ELF. For CK-18, the conventional single-cut-off model produced an area under the curve (AUC) of 0.755 (Figure 1a) while the multiple-cut-off model produced an AUC of 0.765 (Figure 1b). For ELF, the AUC was 0.878 for the conventional model (Figure 2a) and 0.887 for the multiple-cut-off model (Figure 2b).
Conclusions: while the estimated AUC did not differ between the two models, the new multiple-cut-off method allows reviewers to utilize more data and reduces subjective selection of a single cut-off value. In addition, it allows exploration of the diagnostic accuracy of alternative cut-offs, not necessarily evaluated in the original studies.
Patient or healthcare consumer involvement: systematic reviews and meta-analyses are indispensable tools for guiding evidence-based decision making. Evaluation of different analytical methods is necessary to produce robust evidence and reliable recommendations for clinical practice.
Objectives: to compare the alternative multiple-cut-off strategy against the recommended single-cut-off strategy in meta-analysis of diagnostic accuracy studies.
Methods: we rely on two systematic reviews of biomarkers among adults with non-alcoholic fatty liver disease (NAFLD): cytokeratin-18 M30 fragment (CK-18) and the Enhanced Liver Fibrosis (ELF) score. We performed a comprehensive search in five databases to identify diagnostic accuracy studies of CK-18 and ELF in detecting non-alcoholic steatohepatitis or advanced fibrosis among adults with NAFLD. We conducted two separate meta-analyses per biomarker: the first included a single cut-off value, the second included all cut-off values reported in each study.
Results: we included a total of 18 studies in the meta-analysis for CK-18 and nine for ELF. Under the multiple-cut-off model, we considered 33 cut-off values for CK-18 and 16 for ELF. For CK-18, the conventional single-cut-off model produced an area under the curve (AUC) of 0.755 (Figure 1a) while the multiple-cut-off model produced an AUC of 0.765 (Figure 1b). For ELF, the AUC was 0.878 for the conventional model (Figure 2a) and 0.887 for the multiple-cut-off model (Figure 2b).
Conclusions: while the estimated AUC did not differ between the two models, the new multiple-cut-off method allows reviewers to utilize more data and reduces subjective selection of a single cut-off value. In addition, it allows exploration of the diagnostic accuracy of alternative cut-offs, not necessarily evaluated in the original studies.
Patient or healthcare consumer involvement: systematic reviews and meta-analyses are indispensable tools for guiding evidence-based decision making. Evaluation of different analytical methods is necessary to produce robust evidence and reliable recommendations for clinical practice.