AI-Powered Dialogues with Scientific Literature: A Novel Approach to Risk of Bias Analysis Using ROB 2 in PDF Articles

Article type
Authors
Pedrosa M1, da Silva A2, Maruyama E3, de Matos Alves W3, Sugino R3, de Souza G3, Ponciano J3, Feitosa A1, Moça V4
1Unifesp - Universidade Federal De São Paulo, São Paulo, São Paulo, Brazil
2Critical Care Unit, Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil
3Universidade Santo Amaro, UNISA, São Paulo, São Paulo, Brazil
4Unifesp - Universidade Federal De São Paulo, São Paulo, São Paulo, Brazil; Cochrane Brazil, São Paulo, São Paulo, Brazil
Abstract
"Risk of bias (RoB) assessment is pivotal in the systematic review and meta-analysis processes, serving to ascertain the reliability and validity of study findings. Traditionally, RoB analysis demands extensive manual labour to extract and evaluate pertinent information from scientific publications, a procedure known for its complexity and time-intensive nature. Revised Cochrane Risk-of-Bias Tool for Randomized Trials (RoB 2) is a benchmark instrument in this domain, necessitating detailed responses across various domains and outcomes. study introduces an innovative methodology that harnesses artificial intelligence (AI) to streamline the risk of bias analysis process using RoB 2 methodology.
Methods: Our research encompasses the development of an AI-driven interactive dialogue system designed to autonomously parse PDF documents, identify, and extract critical data, and articulate responses to RoB 2 inquiries in natural language. system further users by allowing feedback submission, posing questions for clarification, and soliciting explanations regarding its generated responses. Results: efficacy of our proposed AI-driven solution was rigorously evaluated by applying it to a specific task and comparing its performance with that of a conventional system grounded in primary keyword search methodologies. comparative analysis, thoroughly documented in accompanying table, incorporates insights from a critical Cochrane reference, Physical exercise for people with Parkinson’s disease: a systematic review and network meta-analysis (Review). our evaluation draws upon findings from a pertinent article, ""Six-Month Community-Based Brisk Walking and Balance Exercise Alleviates Motor Symptoms and Promotes Functions in People with Parkinson’s Disease: Randomized Controlled Trial,"" by Margaret K.Y. Mak and Irene S.K. Wong-Yu. This inclusion not only enriches our analysis but also situates our AI-enhanced approach within the context of existing, high-quality research on Parkinson’s disease, demonstrating its potential to streamline and enhance the accuracy of risk of bias assessments. Conclusion: performance metrics, including accuracy, completeness, coherence of responses, user satisfaction and perceived utility, were employed to gauge the effectiveness of both systems. findings reveal that the AI-enhanced dialogue system surpasses traditional keyword-based model across all evaluated parameters. results underscore the transformative potential of AI-facilitated dialogues in conducting RoB analysis, leveraging the RoB 2 framework, thereby setting a new precedent for efficiency and precision in scientific research evaluation.

"