Evaluating the effectiveness, accuracy, and efficiency of traditional versus AI-assisted systematic reviews

Article type
Authors
Gilligan C1, Mulugeta H2, Newby D3, Sinclair P4, Wilson A5
1University of Newcastle, Newcastle, NSW, Australia; Bond University, Gold Coast, QLD, Australia
2University of Technology Sydney (UTS), Sydney, NSW, Australia; Debre Markos University, Debre Markos, Amhara Region, Ethiopia
3University of Newcastle, Newcastle, NSW, Australia
4University of Technology Sydney (UTS), Sydney, NSW, Australia; University of Newcastle, Newcastle, NSW, Australia
5University of Technology Sydney (UTS), Sydney, NSW, Australia
Abstract
Background: Systematic reviews are pivotal in health care research, offering comprehensive insights into prevalent health issues. How will artificial intelligence (AI) impact the process of conducting systematic reviews? A traditional systematic review published in 2023 by Mulugeta, Sinclair, and Wilson, focusing on depression prevalence and its impact on health-related quality of life (HRQoL) in patients with heart failure in low- and middle-income countries (LMICs), serves as a baseline for comparison with AI-assisted methodologies.
Objectives: This study aims to compare the effectiveness, accuracy, and efficiency of traditional and AI-assisted systematic review methodologies, using the depression prevalence and HRQoL in patients with heart failure in LMICs as a case study.
Methods: We replicate the systematic review using AI-assisted approaches, focusing on literature search, study selection, data extraction, and synthesis phases. The comparison will assess time efficiency, comprehensiveness of literature captured, accuracy in study selection and data extraction, and potential biases.
Results: Preliminary results highlight differences in process efficiency, the scope of literature reviewed, and potential discrepancies in outcomes between traditional and AI-assisted reviews.
Conclusions: By directly comparing these methodologies on a specific systematic review, this study provides valuable insights into the benefits and limitations of AI-assisted systematic reviews, particularly in terms of enhancing research efficiency without compromising quality and accuracy.