Article type
Abstract
Background: Artificial intelligence (AI), including generative AI and machine learning tools, may improve the efficiency of evidence review delivery. However, limited research has been done to understand the potential impacts of increased automation on professionals completing evidence reviews, and what challenges and opportunities they may perceive from the introduction of automation into the evidence review process.
Objectives: To assess the perception of automation with AI among professionals who conduct evidence reviews, examining perceived changes in skill sets, roles, work satisfaction, and potential barriers to use
Methods: Staff working as part of evidence review teams within the UK Health Security Agency will be invited to take part in focus groups and semistructured interviews. These will be used to qualitatively assess subjective experiences, including the perception of accuracy, reliability, and expected changes in their professional roles and skills. Thematic analyses will be used to identify patterns of responses within the data and highlight the most frequently perceived advantages, limitations, concerns, and changes that may come from the use of AI.
Results: While the potential benefits of AI on evidence review processes have been evaluated, there has been limited exploration on how the introduction of these technologies may affect those conducting evidence reviews. The nuanced perception of these tools will be qualitatively assessed, and, based on initial discussions, we expect topics to be themed around workload, skill and knowledge retention, working through change, job safety, and trust in evidence synthesis produced by AI. Results from this work will provide a foundation for developing an evaluation framework to assess the impact of AI adoption on professionals who conduct evidence reviews.
Implications for practice: Integrating AI tools into the evidence review process presents a complex interplay between human expertise and technological advances. While automation offers opportunities for greater efficiency, the potential impact of implementing AI on professionals who conduct evidence reviews needs to be identified and addressed. The results of this exploratory pilot study will inform the development of an evaluation framework of the human impact of AI and potentially support guidelines for the implementation of AI within the evidence review process.
Objectives: To assess the perception of automation with AI among professionals who conduct evidence reviews, examining perceived changes in skill sets, roles, work satisfaction, and potential barriers to use
Methods: Staff working as part of evidence review teams within the UK Health Security Agency will be invited to take part in focus groups and semistructured interviews. These will be used to qualitatively assess subjective experiences, including the perception of accuracy, reliability, and expected changes in their professional roles and skills. Thematic analyses will be used to identify patterns of responses within the data and highlight the most frequently perceived advantages, limitations, concerns, and changes that may come from the use of AI.
Results: While the potential benefits of AI on evidence review processes have been evaluated, there has been limited exploration on how the introduction of these technologies may affect those conducting evidence reviews. The nuanced perception of these tools will be qualitatively assessed, and, based on initial discussions, we expect topics to be themed around workload, skill and knowledge retention, working through change, job safety, and trust in evidence synthesis produced by AI. Results from this work will provide a foundation for developing an evaluation framework to assess the impact of AI adoption on professionals who conduct evidence reviews.
Implications for practice: Integrating AI tools into the evidence review process presents a complex interplay between human expertise and technological advances. While automation offers opportunities for greater efficiency, the potential impact of implementing AI on professionals who conduct evidence reviews needs to be identified and addressed. The results of this exploratory pilot study will inform the development of an evaluation framework of the human impact of AI and potentially support guidelines for the implementation of AI within the evidence review process.